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I. INTRODUCTION •' • . , 

Compensatory'^eduqatlon became'a; concern of primary .interest 
^ nearly a. decade after the Untied Statfes Supreme Colirt cjecision 
in Brown vs Topeka when public school officials were untler 
massiye pressure to Improve the education of impoverished black 
children. In the" eaMy I960' s,- pointed arguments by 
J. McVieker-Hunt" (1961) an.d seductive data frqin enrichment- type 
prdgrams; (mostly supported by private foundations) directed by 
, sucK reslearcheirs as Martin Deutsch and Susdn Gray seemingly put 
tOTest' the long-espoused dpctrine that disadvantaged racial 
and' 'social -class groups of people were* poorly endowed by nature. 

, vftcademiC failure and progressive retardation, the 
cumulative deficit '(Deutsch, T9670 found s6 frequently among 
^pobr children, led to the^cu^Uufal deprivation hypothesis, 
a.causal argument based on early'soeialization under conditions 
of poverty and diiscrimj nation. -While Interest in this 
hypothesis i> still much 'in evidence, it was challenged success*- 
fully by educator| who laid the blame for academic failures 
among .disa.dvantaged children to "grossly inept practices in the 
schools" (WiTkers6nrT970)., ■ 

"In 1965^ ■ Congress passed the Elementary and Secondary 
Education Act. Titlje I of this legislation represented the 
first. major aVloca'tion of funds aimed toward closing the 
educational gap between the disadvantaged and the non- 



'disadvantaged. Title I ESEA was not the only compensiatory 
educatton effort of the federal government, however. Dis- 
advantaged students are also aided under such programs as 
Head Stairt and Follow Through. While compensatory programs 
have varted considerably with respect to what the funds have been 
expended upon, they have all shared the" common notion that the 
funds were to h^lp close the achievement gap as measured by 
generally accepted outcome measures. 

During the early years, federal compensatory funds were 
used as a general aid to education with little attention to 
how the funds were directed to the target population or what 
use was made of the funds, is the Senate Select Committee on 
Equality of Educational Opportunity expressed itself: 

... in a real sense, compensatory education has 
never had a 'chance—servi.ces have, often been , 
diluted to the point of meaningless" -(sic) r ' - ' • .. 
and even extended to non-el iglbte children: . , ' ^ 
Tunds have been expended on -equipment which- 
- \ .Mis never put" to, effective use. - • . 

Equitable .and. efficient procedure^ -to assure that -federal funds' 

are expended' upon the target population for the purpose intended 

have been dtfficulfto achieve, at both' the federal and local 

-levels. .A signlficafht^ effort by the ^epngress to , effect this ' 

assurance is contained.-tn, Jhe Educat1on"Amendments of 1970. ^ 

According .to th.1s legislation, schopl d1str1-cts which receive-- 

funds^ under Title I for. children of '1 ow-1 ncome families must , 

first use state and local fufids to, provide services which are » 

comparable , at least, tg services being ^)rovided by districts/ 



which are not receiving funds under Title i: Implementation 
of Congress' i n ten tioit would have'the effect of assuring 
that districts receiving. Title I funds will regard these 
funds as supplementary and. for the purpose of mounting special 
attacks upjDn the educational deficits of compensatory students. 

At the school -level , ^ former requirements that Title I 
funds be used exclusively for the benefit of compensatory 
students created special problems for the delivery-of in- 
struction. Prevalent practices employed by schools, to- 
accommodate the exclusivity requirement Included: Cl).sprecial 
in-class tutors, (2) "pull out" of compensatory students from 
mainstream classes for special instruction, and (3) formation 
of homogeneously grouped classes of compensatory stu^lents, 

It^has been argued, however, that discriminant provision 
of special Instruction for compensatory students has the effect 
of label.ing them as' "disadvantaged" with consequent negative, 
effects (White et-.al., 1$73). Oh the other hand, not limiting - 
services to targetecl students has the potential disa'dvantjige 
of spreading funds so thinly that no educational Effects • 
could be reasonably expecte^l. ^ ^ * 

* Equally as troublesome as the problem of targeting funds 
was the determination of What needs to. be done with available 
funds to raise the educational achievement of compensatory 
students ta a par. with their non-compensatory counterparts. 
Despite the generally poor* caliber of studies of compensatory 
education, s^everal facts have teen common to studies of ^ 
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successful compensatory education programs'^Ce.g. , Wargo et.al., 
197V and. Holzm^n and Boes, 1973). These studies have observed 
that the shared characteristics of successful prlpgrams have 
been careful planning, Including clear statement of objectives; 
teacher training iri the method of the program; parent involve^ 
ment; high Intensity of treatment; and Individualization of 
instruction.. The latter characteristic, individualized 
instruction, is the focus of this study. 

If the evidence were clearly convincing that individualized 
instruction was the most effective instructional rtode for 
compensatory education students (i.er, if students receiving 
such instruction achieved significantly more than similar 
students taught in other ways) then individualization could be 
Viewed as a singular splution to both the, problem of 
separating (labeling) compensatory education students for 
differential instruction and the problem of instructional 
effectiverless. , - . • 

However, apart frpm the question of compensatory education, 
the overall evidence on the effectiveness ^f individualized 
instruction is ambiguous; i-e., positive effects have not been - 
consistent over. time and place. Moreover,- the meantng of the 
term "individualization" is itself in considerable dispute. ^So 
many programs which, in* many ways, are Tad'icajly different 
have been advanced under the label that, according to Gibbons 
(1970), thje term has lost its value as a' useful xategorizer. 
The ambiguities of both meaning and effects may be related. 
Indeed, it can be argued persuasively that'the variation In 
outcome effectiveness of individualized instruction is, in 
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fact, a reflection of its various characteristics in various 
contexts as it is employed for a variety of purposes. 

Studies of individualization and compensatory education 
hive suffered most of the same flaws that have marred most 
studies of -educational Innovation In the past. The flaws that 
we have obseWed to be particularly pervasive are the 
following: * 

1, Failure to ascertain tHe exact nature of 
the Intended treatment or treatments. 

2, Failure to ascertain- the exact nature of 
the Intended outcomes of the treatments, 

* 3, * Failure to verify that the intended treaty 
ments were In place (and not confounded 
with comparative treatments) during the 
period of evaluation, 

4. Failure to. meet the technical conditions 
under which summative judgments of effects 
could be validjy made, . 

5. Failure to provide s'uffi-cient opportunity 
for maximization of treatment-outcomes 
variance \to occur, 

6. Failure to employ criterion instruments 
•that would be consistent with 'intended 

effects in terms of kind and scope,' - * 

7. Failure to specify the, conditions which 
qualify the treatment-outcome relationships . 
in specific contexts. 

Thus far, we have -briefly surveyed the Intentions of the 
Congress with respect to xompensatory education, 'the ^diffi- 
culties th.at have* hampered 'their implementation, and the ^history 
of inadequate evaluations that have deprived the Congress of 
the Information, necessary for an enlightened review of . , 

■ in. .. 



compensatory policy effectiveness, and a useful guide to policy 

« « • • • 

reformulation. Qlven this state of affairs. Congress in- 
cluded in the Educational Amendments of 1974 instructions to * . 
tKe National Institute of -Education-to design and conduct .* 
a comprehensive study of ^compensatory education. 

On the basis of this new research, Congress intenlds to draw 
.conclusions about the fieed for further legislation and programs 
in compensatory education, and to use the results In Its ' 
deliberations in 1977*, concerning reauthorization of Title J 
of the Elementary and Secondary Education Act. The overall 
itivestigation is to be ver:y broad, but the study design de- 
scribed In the body of this document Is only. Intended to pro- 
vide evidence- about "the effectiveness of -Individualized ' 
instruction a,s it' is currently used by corrtperisatory pro'grams 
in schools'* according to RFP NIE-R-75-0022. The RFP further ' 
states that: . ' ' / ' . ^ ^ 

• ' , The study is not meant to be' a comprehensive, 

survey but rather a concentrated examination of '\ 
'selected programs which* provide individualized ' ' 
classroom instruction. The major focu^ of the 
study will be on assessing program success in 
^ the areas of ^reading and' mathematics achievement. 
•In addition, however, some more general im- 
plications of individual iz6d instruction for the 
\ classroom environmervt will be examined. 

The study we have designed is directly responsive to 

tjie concerns of the NIE as expressed in the RFP, and great 

care has been taken to attend to the problems and flaws , of 

research evaluation that have been noted above. Howevef, our 

experience in developing and evaluating, programs of"' 
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1 nd'1v1 dual 1i!ed Instruction leads us to believe that m. 
strong. Evidence of the effectiveness of Indlvi dualized 
' instruction is apt to be' discerned from a slmo-le com- 
parison of the effects of "Individualized Instruction" 
wi^h the effects of "sta ndardized instruction. " " 
• The rnultiplictty of 1nstruc,tion/il. pllails' grouped under ' V 
tftff label '"Individualized", vary. widely- In the fr emphases, 
and, frequently, thefr. operational interpretations of the ' 
.term contradict each other to the^nd that significant effects! 
of soiJie p.lans likely ^I'M be masked by offsetting effects of 
others. Moreover, some features of instruction formerly 
associated only with* individualized instruction have been 
^adopted and/or adapted for use in standardized classrooms 
^such'thit individualized and standardized treatmerrts of in- 
structfon? are somewhat confounded, and this precludes any 
clear discrimination of instructional treatment effects by' 
direct companison of labeled groups. r 

• We are in agreement with Jamison, Suppes and Wells ^ 
(1974) that what is needed is to ascertain more exactly the' 
nature of conditions that do make significant particular 
factors of instruction. The study should produce findings that 
relate. outcome effects to 'ti;eatment varia^tions under specified 
conditions. Thus, we have- gone, beyond the direct comparison 
question in designing a study that willHeliver information 
„ that should be critically important to NIE and the Congress. 
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The following overvtew of the study design highlights 
the principal features of the study and th?* rationale that 

* ♦ 

decided their inclusion* . • ' 
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■ II. OVERVIEW OF THE STUDY PES 1 6H 

The study des.cptbed tn tftts iocument h«s been design 
to fit the stucly requtretnents .stipulated by the NIE-RFP. Every 
effort has be_en-inade-to; acconmodate the many constraints and 
principal Issues fdehttfted by the NIE. 

Design requlreTOnts -^The NIE Intended that the study design 
•for-^a»i\r|irn;prov1de the means of investigating the contention 
that Individual 1 zed Instruction Is especially effective in pro- 
mpting achievement gains In compensatory situations. It Was 
^specif fed that -a . comparatl ^ stucly be designed "to assess^ the 
effectl^ness^ of individualized fnstruction as it is eiinrently ^ 
u«d ^ compensatory programs' in schools" and thatjistwdardl^ed - 
^instruction" be used for comparison purposes..^ ^-^^..I--^- . 

-Also,, study conditions should conform with the requirenent that 
1:he effectiveness comparison Involve only ''weVl-implemQnteA 
programs of , both types" and that "in-depth observation dfpro-v 
grams" be made "to fullj describe their operating character- 
Istlcs, to determine degree of implementation, and to describe . 
settings where adequate implementation may be difficult to 
achieve." 

. Finally, the NIE required that the effectiveness of programs . 

"be assessed by measures of reading and mathematics and by 

«- 

(program) effects on the classroom envi ronnjent . " - ' * , 

Amohg the design Issues of special Importance to the NIE weret 

• establishjnent of criteria for defining programs 
as Indtvtdualized or standardized; 



* . ' \ 9 ^ techniques to^be used in assessing program ^ 

\ / imp! ementatfon both for selection and validation 
purposes; ^ . * ' , 

V f; the development or identifi-cStLon of measures' 
' for assessing the qua^l ity. of 1 implement a ti on 
along with procedures for »determinio'g*the^t 
. ^ reliab-HUy of such '4D6asur«s:f ; . . . \ , - ' 

* , * • procedures for relating, differences in level of 

iiflplementation to both plro'gran\ and outcome \ 
variables; . . ' . 

• a selecticm procedure that provided for systematic 
/sampling of programs' along an instructional setting 
> dimension; and » , . ; , 

' , ' » ■* * • 

0 the idehtifi cation' of criterion measurers for 
\. achievement with appropriate »testinq' strategies. 

A. tHE>^)isiGN : " ^ ' - ° 

In order to be fully, responsive to NlE's concerns we hav^ 

developed fburjiiajor study questions with appropriate specifications^ 

and -procedures for, addressing each. ^Ttw Study Questions are: \ ' ^ . 

(1) ^What ia the relative effeativeheaa of Dell-'i^mplemented 

Hndividualized" inatTuotiony^- w^il-implemented 
'^atandardized'^ iiistvuotidn for recxipiente of ophrpen" 
eatory education in teima of reading aohievementj 
mathematioa aohievement and olaaaroom olimate? 

when the reaipienta of ^oompematpvy 

education receive inatmotion as a ae'^araie group and ^ 
i , when they are inatruated along with non-targeted' 
ohitdren? ' ' ' ^' ^ 

What ia th'e relationship of degree of implementation^ 
of various programs to student outcome variables? 

(2) ^Hm does spedfioity of treatment, of the instruational 

taskSj taken' one at a time and in sequentiat odTfibina- 
tions^ relate to cognitive and affective outcomes? ' 

How does treatment of the instructionqfi tasks together 
with classroom climate relate to cognitive and 
affective outcomes? 
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(3) Sow doee epeoffiai^ of the ireatmenf of 
. imtniot-Coml taeko taken along with other 

VaHables of aonaeim relate to student 
achievement? . " 

(4) Whit- are the oonditiow that oontrihite to 
differences in degree of ^lementation? 

The ftrst study questtoh ts intended' to be a direct rep 
resentatlon of NIE's interests as stated tn the RFP. The 
question implies a straightforward summ'ative approach to de- 
terniintng which of two instructional techniques are more ef- 
. -fectiye for increasing achievement of compensatory students. 
^ Jhe comparison can be made survey-style, assessing student 
achtevenient at a single point in ttme, categorizing ..ipstruc- 
tibnal programs as well -implemented Individualized or well- 
tmplemented standardized^rograms, mainstream or separate, and ' 
making the required comparison. The approach ts direct and un- " 
complicated and' may provide straightforward and unequivocal in- 
formation to the Congress. For this reason, we havejncluded ' 
.in the study design the- means for responding to the effectiveness 
.. comparison question as stated and have Included two subquestions 
which reflect the specific concerns of the NIE regarding the ^ . 

effectiveness of mainstream vs separate instruction and the re- 
lationship of degree, of program implementation to student outcomes. 

The effectiveness comparisons of Study Question One (parts' a 
and b) requ^ little more than 4)ost treatment testing on the 
'criterion. measures of student perceived classroom climate and 
reading and mathematics achievement; a means of identifying programs 
as individualized or standardized; a means of Identifying well- 
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tmplemented programs of both tnstructtonal types; and the' 
Identtftcatton of mainstream and separate tnstructtonal settings. 

. In order to achteve a dtchdtomtzatton of program types we 
have recommended that tnstructtonal plans be categortzed as 
rindtvtdualtz^d or standardtzed on the basts of tnstructlonal 
untt stze attended to for each of a common set of Instructional 
tasks. 'We have recommended a standardi;2ed achievement test that 
•can bej used in. a way to> reduce "floor and ceiling" effects. We 
have also recommended use of criterion referenced exercises from 
the National .Assessment of Educational Progress for*the derivation 
of special comparison' Information' that is unique with these items. 
The analysis recommended foir investigating the effectiveness 
y^^f indlviduaTized or standardized instruction and mainstream vs^ 
^parate instructional settings is a\2 (program) x 2 (setting); 
fac^torial design employing univariate and multivariate analyses 
jDf covariance (student IQ and SES) on* feadingi mathematics and 
student perceived classroom climate. ^ - 

Since it is not clear who the Congress or the NIE. refers to 
as compensatory students we have recommended , that the analysis be 
carried out separately for^students qualifying on*(l).a poverty , 
criterion, (2) an educattonal^ need crtterion, arid (3^ both 
groups together* Degree of implementation in the general "indi-- 
vidualized" and "standardized" cases we have arbitrarily defined 
as consistency of treatment over time. . 

We are reasonably certain, however straightforward and 
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unequivocal the inforrnation obtained froni such a- comparison 
may seem to be, that the findings from such. a comparison will ^ •' ' 
not be very meaningful without further specification of tKe 
underlying differences among programs claiming to be indiv- 
idualized and among programs claiming to be stahdardfied and 
■of the overlap that exists between programs of both types. For 
this reason, we have developed Study Question Two that require^ 
an extension of the design for Study Question One and Velates- 
variation of instructional task treatment with outcomes , regaVd- 
less of program labels. That is, labels "individualized" and • 
"standardized" will be ignored. Study Question Two, then, 
requires consideration of the same set jof instructional tasks 

t 

discussed under Question One, but whereas the tasks in Study . 
Question One were looked at only in terms of instructional unit 
size, this time treatment of the instructional tasks is examined 
in terms of (1) instructional unit size and (2) the level of 
attention given the task. Under Study Question Two It will be 
necessary to relate student gain scores to climate measures and 
instructional task treatment^. 

• Three analyses are proposed for this part of the design:, 
commonality analysis (Mood, 1971), a series of stepdown 
regression ^analyses and canonical correlation. 

Study Question Three has been deyeloped in response to the 
NIE's request that study plans include provision for determining ' 
effects of program variables together with other variables and 
"include specificatipns for arguinWausJil relationships between 
program and outcome variables." We have set out such specifications 
in a hypothetical path model. Th^odel contains nodes that include 
sets of composite variables derived from the total set of variables 



used>in the study design,^ Procedures %havfe |)een specified 

for selecting constituent variables for a priori; confposites \ 

and subcomposltes for assessing the poweV of the model 

to €xpla-in variaace jn Student achlevisment. '• , " 

' . ■, ■ • -■ • 

' ' ' Finally, Study Question Four was' included to provide 

the means for examining thos^ conditions that may make it 

difficult to achieve good implementation. / ^/ ^, 

While It .might seem that we have designed four separate 
studies, most cleSign features that we have selected or developed 
have value across study questions- For the sake of clarity, we 
have organised this document to treat the stqdy questions inde- 
pendently. However,* the majority of the- design featuWs are \ 
discussed uflder Study Question One and^only differences and 
additions are noted under subsequent questions. 

Some' principal features of the design are discussed below^ 
along with discussion of some of the, design issues of special 
interest to the NIE. , ^^ \ 

Defining individualized and standardized instruction - We 
hiive chosen , to define program t^^pes in terms, of 3 set of common 
instructional tasks. The tasks are defined under the Rationale 
for Selection frf Variables, Definitions and Instrumentation 
sectfpn of Question One..' Other specific design, issues dealt 
'With in this section are criteria for defining^^programs as 
individualized and standardized instruction, techniques to be 
used In assessing program implementation, and the criteria*^to 
be used in attributing the quality '^wel 1 -implemented" *to an 



•instructional program. ' Instructional task treatments are fully ' 
^ V- . . . • 

deftned under the Study Variable' section of Study Question Two. 

^ • Assessing tmpl^nientflitton - We have produced the tnstru- * /. 
mentation for assessing program implementation at both the 
selection and observattoh stages of the study. Preliminary 
tryouts of the instrument indicate- that with minimal training - 
experienced curriculum ?taff are able to use, the measure with 
good consistency over treatments types. For the study itself 
we recommend that training be continued until inter-rater re- 
liabilities reach a mtnifum of \80 for simulated and'on-site - 
observations. ^ Suggested procedures for traiging are contained 
in the Appendix. Procedures tov be used in relating degree of 
implementation to outcomes are. discussed'in, the analysis section 
for Study Question Ojie, part c. ' ' ^ - 

/ Sampling and selection - The sampling and seilecti on design 
was structured tb sample systematicallyv along all busic dimensions 
including the Instructional setting dimension.. We anticipate 
that real world complexities will make it difficult to' irfentify* 
settings as either mainstream or separate instruction. The de- 
finitions we have provided for this purpose should be useful In 
making the necessary distinctions. They are located -in th^ 
Sampling and Selection section- under Study ijuestion One. Most 
-of the selection procedures tiave been trie^d out an^ rej/isi)ons 
have been made on interview forms where they were needed^. rAn 
forms, to be used in selection are to be found in the Appendix. 



Criterion ajcMevement ifteasures- We have Tecomrienderf the ^ 
use of both th e California Achievement Test and sel ected items 
from the National Assessment of Educational Progress... -The* 
features that we feel make eeicK a desirable' choice, fon this 
study , are discussed under the 'Question One section "Rationale 
far Selection of Variables, Definitions and Instrum^tation." 

Analysis- The analytic procedure reCornmended for ; 
Study Qiiestipn One is a 2 x 2 factorial design employing 
univariate and multivariate analysis of co.variance (IQ and SES)^ 
on reading*, mathemattqs .and student perceived classroom climate. 
This analytic procedure we recftmmend should be performed - 
separately* for students Identified as compensatory on (1) a 
fioverty critemiv^ (2) ao educational criterion/ and (3) both 
groups , together/ , ' , ' ^ 

For Study Question Two, three analyses are recommended. 
They are coninipnallty apalysis (Mooc^,. 1971), stepdown regression 
anarlyses and canpnic&Vcbrrelatlon. Under Study Question 
Three, we recommend the method of positioning variance in 
multiple regression analysis; stepwise ihegression. analysis ; 
factor analysis and path analysis. Multiple discriminant 
analysis on predetermined groups is recommended fdr Study 
Question Four. 
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,111. METHODOLOGY 

A. STUDY QUESTION ONE 

1. Rationale for Selection of Variables, Definitions 
and Instrumentation [ 

Based upon the directions and specifications set out by the 
NIE in its RFP, it seems clear to us that the study question of 
main interest ii the following. 

a. What ie the relative effeotiveneae of well-- 
implemented '^individualized'' instruction ve, 
welUimplemented Standardized" inatruation 
for reaipiente of compensatory education 

in ierme of reading achievement^ mathmatios 
achievement and classroom climate? 

b. ^ when the reci^pienta of compeneatcry 

education are instructed as a separate group 
and when they are instructed along with non- 
targeted children? 

C. Vhat is the relationship of degree of imple- 
mentation to outcome variables? 

Essentially » Question One focuses upon assessment and com- 
parison of the outcomes of two instructional prpgrani types. It 
is implicit in the question that outcome effects are to be 
attributed in this question to the respective instructionaT pro- 
gram types with but two <iuaTifying conditions^ instructional setti 
and degree of program implementation. While the labels suggest 
easily identifiable instructional settings, "mainstream" and 
"separate/' the complexity- of the real world of schools belies 
this simplicity. Discussion of our strategy for identification 
^of sittings is reserved to the section on sampling and selection 
procedures. The remaining variables of principal interest in this 
question are defined and discussed^below. . 
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Instructional program in this study design refers to the 
practices, procedures and materials of instruction that are 
^articulated in some systematflc way to enhance the ^achievement 
by learners of some intended set of outcomes. Programs differ 
1n terms of* their processes and materials for attending to a 
set of generic instructional tasks; but it is our view of the 
classroom that teachers and programs, whatever their. labels, 
give some level of attention, more or less specific, to each 
of these tasks for each student, more or less specifically. 
We rccownend that these tasks constitute the conmon set of 
program Variables for identifying' "individualized" and 
"standardized" programs of instruction and for assessing their 
degrees of implementation. 

Glaser (1963) identified a set of basic Instructional 
tasks which we have modified and elaborated for purposes of this 
design. Modification and elaboration were guided by our concern 
that program variables be adequately comprehensive and repre- 
sented by moderately-high inference observations. The instruc- 
tional tasks (program variables) recortmended are: 

1. Provision of Curriculum Opportunity 

2. '* Statements of Curriculum Intentions 

3. Curriculum Placement Decision 
• . ' 4. Adjusting Rate of Instruction 

- 5, Provision for Individual Responding 

^ 6. Provision for Indiv.idual Feedback ' 

7. Monitoring Individual Progress 

^ 8. Performance Standard for Advancement \ 
% - ■ " 

9. ^valuation of Performance^ 

lOj Matching Learners with Next Instruction 
111-2^ 
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Provision of Curriculum; Opportunity refers to the 
opportunity for learning that "is provided for learners. 
■Whether or not the opportunity Is adequate for each learner,^ 
individually,' win depend Upon the- relevance of what Is 
offered, the 1 earner's extant state of achievement and the 
learner's potential for growth during the school year. 

Statements of Currlcular Intentions describe potential 
outcomes of instruction and, learning. *They can be stated 
for individuals, variable subgroups, or the class group 
as a whole. They may simply state what is to be- offered 
and not what is to be accomplished by whom. In this event, 
the teacher makes the decisions about t& whom they are to 
apply and to what extent. . 

Currlcular Placement Decision Is the task that deter- 
mines the point-of-entry Into the curriculum for the indiv- 
idual, the variable subgroups of the class, or the class 
group as a whole. . - " , 

Adjusting Rate of Instruction , wherever the locus 
of control, regulates the rate at which new content for 
learning is made available to learners. The rate may or. 
may not be appropriate for anyone but, nevertheless. It 
will be set for Individuals, Individually; or variable 
subgroups, separately; or for. the. class group, as a whole. 

« 

♦ Provision for Individual- 'Responding by students 
implements the assertions of;.Piaget and behav tori's ts that 
learners learn by acting upon whafls t6„be learned. Although 
group respondifig does oqcur. Individual responding ;is 

- ' •' ,. ■• -'' ll l*3 • ■ ; • , , • . 
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thi dominant* mode 1n both Individualized and" standardized 
InstVuction. What is additionally important and the 
aspect- of concern hefe is the frequency of response 
opportunity for individuals. One expects that individ- 
ualization affords greater frequency of o^rtunity for 
individuals, to respond as Individuals than does standard- 
ized instruction. 

Provision for Individual Feedback includes all kinds 
of information directed to the learner relative to the 
learner's progress. Accuracy of responses, identification 
of errors, cues, suggestions and leading questions are 
all examples of informational feedback. Feedback may 
be oral or in writing. As with responding, it is recog- . 
nized that feedback' is sometimes given to groups but , is 
mostly intended .for individual use even when offered as 
a general ^announcement^, ^t is generally assumed that * 
individualization prdvides for mor^ frequent feedback-- — 
which is the concern here. 

, Monitoring Individual Progress through the content for 
learning takes the form of frequent checks of accuracy of 

pupil's work samples » present position of pupil in the % 

\ 

Instrucftonal sequence; adequacy of pupil's rate of .pt^ogress 
evidence from a variety of sources of pupil 's need for 
assistance. With this task> also> the concern is one of 
frequency . . 

Perf orihance Standard for Advancement may be applied to 
each Individual in the class; or to variable subgroups as 
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groups; or to the. class as a wholfi. The' standard is' the 
level of performance that must be attained by the learners; 
e.g., 80% or better on a^task. 

Evaluation of Performance is a judgment of the per- ^ 
formance of learners relative to^ some standard of expectation. 
Performances of learners may be judged in ternis of the mean 
performance of the class or variable subgroup, or perform- 
ances of indfividuals may be judged individua^lly. ^ 

, Watching Learners with Next Instruction can be arranged 
for individuals individually, or for variable subgroups of 
the class separately of the class as; a. whole. Matching • 
presumes' some kind of assessment of the learner's needs and 
provision of appropriate instruction. Assessment and 
matching may be by teacher or learner and may be overt or 
unobtrusive. 

It should be noted that, while some additional tasks 
(variables) could logically be included in our program 
list (e.g.> curriculum sequence), our experience indicates 
that determination of specificity of treatment, for those . 
we have deliberately excluded, requires very high inference 
observati^ons for which inter-observer reliability is verjy 
low. f'^' 
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Ea.ch of the program variables vartes in terms of the 
specificity of treatment It receives during the Inslyructlonal 
process. Specificity of treatment c4n be regarded as loosely • 
analogous to the vector sum of the (llToenslons: ' Instructional 
unit sdze and l^ewi^of attention . • . 

Most educators will agree that a<:h1evement Vikely will be 
greatest when learners and instruction are optimally matched 
according to the needs and £pm}ietenc1es of the 1 Earners . Indiv- 
idually. However^ educators sfeem to be sharply divided^ as to 
whether the Instructional tasks are most effectively attended 
for learners in Isolation or as memb'ers of a group. Many 
educators even advocate varying the Instructional unit size 
according to ttie content for learning^ the Instructional task 
•and* for different propoHtons of the school day. 

To restrict the Instructional plans that are labelled 

Individualized to only those that attend to the instructional 

tasks for individuals* individually^ would surely be controversial 

and would seriously jeopardize the credibility of the study's 

findings for the thousands of practitioners who frequently 

utilize small group settings 'and* yet* purpoK to be individualized. 

On the other hand* to ignore instructional unit size in defining 

individualized plans would surely invite confounding of Individ- 

« 

uallzatlon and standardization. in this study. 

An alternative would regard indlvidualtzatlon and 
standardization as variable subcomponents of a composite called 
classroom Instruction. This possibility has much mertt» and we 
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win attend to' It under study questions two and three. ^However, 
*..1f the study design. Is confined only .to this alternative It will not 
be responsive to NIE's main question of Interest-. 

Of course. It will also be said that cons.1dera.t1 on of 
Instructional unit size Is Insufficient for determining "best 
fit" between learners and their Instruction, and we must agree.^ 

» • _ * • 

Not only is the degree to which the instructional tasks, are attended 
for learners individually iniportamt> but the kind of atten'tloni 
l£«., the level of attention that is given the" tnstructional task 
for them is also Important. Educators ^re just as divided on 
what l,evel of attention' is. appropriate. -Some argue.'that "Best 
fit" •is-iiiost' effectively. attained by preplanned highly specific 
attention; others are equally convinced that ihe learners are 
most likelA^ to find their own best fit when .tliey are free to dis- 
cover their interest-related needs in relatively unstructured, 

« 

" I' 

broadly specified Instructional contexts. Again, there are many 
who opt for eclectic mixtures of the 'two extremes . All of-wMch 
further complicates the dichotomization of instructional plans 
into ipdlvidualized and standardized categories. * ' \ ; 

It is our recommendation, therefore, that for purposes of • 
question one, ■individualized instruction (individual or 
variable subgroup) and standardized instruction (class group) 
be defined in terms of Instructional unit size only. T\\is is 
an important dimension of task treatment, and consideration 
of its relationship to achievement and classroom climate 
will be appreciated by all practitioners who claim to 
be .individualizing but whp likely would be excluded by 
one or the other of more restrictive definitions. Special 
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attention to questions of how the Influence of Instructional 
unit sl2e and level of attention given to task treatment 
' conjointly relate to outcomes will be taken up in questions two 
and three where the definition of program variables will be 
further elaborated. 

Given that there are ten program variables and site? may 
be identified as either individualized or standardized on each 
task, a decision will need to be made as to how many tasks will ' * 

*' *• require either an indivi dualized or standardized rating for a ' 

site to be clearly qualified as efther one type or. the otKer. 
Ideally, §ites should qualify for their classification on all 

, ten tasks. Realistically, this may leave the follow-on con- 

tractor with Insufficient sites to meerthe requirements for 
analysis In one or both categories. If it is necessary to 
reduce the number of tasks required for qualification, we re- 
commend that "responding," "feedback," and "monitoring" be 
successively removed from the qualification requirement until 
sufficient sites have been identified. While these are Important 
tasks, they are probably equally subscribed tp by both plan 
types, and may not be strong rdiscrimina tors of the comparative 
programs in any event. " 

The RFP stipulates* that the comparison of individualized and 
standardized plans on outcome effects be restricted to " well- 
, implemented" plans of botfi types. For u^. this means that the \^ 
, sites must ifiitially me^t the qualifications for their respective 
labelk; i.e., be'*judged consistently individualized or standardized 
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' ;.in. their treatment (instructional unit size) of the instructional 
tasks (program variables}^ " Ideally, sites should maintain their 
■ qualifications on task treatment across observations, over the. 
course of the school year, to be considered well -implemented; In 
the event that-too few schools retain their qualifications the 
> " contractor shouTd establish a cut-off point (based on the quanti- 
■ fi cation system described on page I I 1-41 for qualification as 
well-implemented. " . • . . 

.Sites that do not maintain their well -ijnplemented class if i- . 
cation need to be' eliminated > from the analysis for ..parts a and b 
of this question. ■ rtowever, not Well -implemented sites will need 
. to be .included along, with we 11 -.implemented sites to detenriine the 
■ relationship between- degree of impletnelntatlon and outcomes^ which 
is the requirement of jpirt -c .of this question. 

• Observers will be,}:ra1ned'to. use' the Instructional Task . 

," .■ * ' - ' • ^ t ' , , ^ • ' . ' 

Treatment Observation -instrument (ITTOF) to make moderately high^ 
• inference judgments of individualized' or standardized -treatments 
of the instructional .tasks. Observers will employ interview 
techniques with teachers and students ^ seek out and interpret 
appropriately indicative "instructional materials and records 
and note their judgments of • instructional unit size (individual, 
variable subgroup or class group)- for each -task. Instructional 
unit size of "individual" student" or "variable subgroup" . 
qualifies the site as individualized for'the relevant task. . To ' 
be variable requires provision for regrouping throughout the , 
year. A "class group" judgment indicates standardised ins.truc- - 
tlon. The strategy for scoring the ITTOF and the decision rule 
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for labeling 1s treated In the data analyslf^ section under this 
quest.lon. 

Instructional programs, as we have narrowly defined them, 
operate, in a classroom cl^lmate characterized by many .interpersonal 
acts and their consequent effects upon teacher and student 

c 

attitudes, and feeltngs. Selection of sites for study under Question 

One requires that the sites meet mfniraal , at least, conditions of 

classroom climate that are deemed necessary for adequate program 

implementation* Soar and Soar have provided a small set of 

Climate variables for screening purposes. The Classroom Environment 

Screen\ng (CE-SX Instrument is discussed in the sampling and 

selection section under this question. 

Outcome variables of interest in question one are as follows: 

• Reading Achievement 
t Mathematics Achievfment 
^ • Classroom Climate 

Despite the dissatisfactions witli standardized tests that 

have been voiced In many quarters, these tests, in lieu of adequate 

substitutes, are still relied upon by lay persons and educational \ 

administrators for assessment of learners* content achievement. 

Since the principal standardized tests have generally high relia- 

blTity and validity, our principal concern in selecting, a ^ . 

standardized achievement /test was the degree to wh1<:h the instru- 

ment would -reduce the problem of "floor and ceiling" characteristics 

that severely liniit the capacity of these instruments to reveal 

-diffeftntia^ performances of many students whose actual achievement' 

levels fall outside the grade:^ppropriate battery range/ 
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The Corapreliensiye Test of Basic Skills (CTBS) and the. 
CallfornU Achievement Test (CAT) are unique among standardized 
tests for their continuous scale scores across test levels" which 
afford the opportunity to test students out-of-leVel. The, 
standard administrations of these tests acc(»nnodate this option. 
We recornroertd that students be assigned to test levels" on the . ; 
basis of available' scores and/ or teacher judgment. 

Our final recoimiendatlon of' the CAT Is Influenced by the 
greater range of Its Level 1 (2.0 - 4.9 vs 2.5 - 4.9 for the 
CTBS) and the substantially shorter period ot-time required for 
Its adn;1n1 strati on (127 minutes compared. w1 th the CTBS' 176 
minutes)'- Given that extensive .testing already joes on in most 

schools, the substantially shorter* testing time required by the 

CAT should helpio qnlist school cooperation for the study. 

^ Additionally, we stro^ngly recommend administration of sets 

of Qtjlterioji-ref^rericed test items, for reading and mathematics, 

selected by us from the poo] released by the National 

Assessment of Education Progress Project (NAEP)l .Use of the 

subs)ets of NAEP items will significantly buttress the cognitive 

testing program in areas where the standardized tests' are 

.flagrantly weak. • Some of the advantages to be gained are: 

(1) NAEP items are' tied directly to 
educational objectives. 

"(2) The objectives have been endorsed.by * 
, scholars, educators, parents and ^ . ' 

other lay persons as important for 
>. , • youth tb know and having- value in 

' modern life. 
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(3) The items are designed .to' be- approp- 
riate for a wide range of ages and 
span a wide, range of difficulty levels. 
(The items are not chosen for their, 
power to discriminate among r^^pon- * 

• . . ■ dents as are standardized tests items.) 

(4) Performance data for NAEP-define(f 
population subgroups and the natfonal 
test population are. available by Item. 

The NAEP descr:ipt1ons of communtties in which "test 
Pesp^ljilents reside-are particularly'relevant for the' 
compensatory aspect of this study. That is, the purpose;.' 
' of compensatory education ,is to close the gap between the •• 
educationally disadvantaged and the norm for all students. 
This is most meaningful .at the'conmunity level." The NAEP. 
Identifies performances by geographic region, inrter cfty,. 
rest of city, suburban fringe, extreme rural, etc. The 
. NAEP also reports for male and female'and black and white 
... races. ,To facilitate comparisons the foll.ow-on contractor ' 
should describe the communities in whtcH the sites are 
Ipcated in' terms of NAEP criteria. 

A performance data summary'sheet supplied by NA£P, ; 
Congressional testimony of W, J. Popham on behalf of ; 
criterign-ref ere need testing, and the item subsets>e haVq 
selected are included in the* appendices. " . ' ' 

Classroom Climate may be perceiVed as a process variable' 
or as an outcome effect of cTassrooiri transactions. Techniques 
of S6ar and Spar for assfessing^ classroom climate, over time have 
* been notably effective, and it is planned that they should tJe 
used in "addressing Questions Two' and Three. However, we perceive 
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climate as an^outcome concern in Question One, and, sfnce"^. 
treating the Soar variables a? outcomes would require assigning 
the same climate: scores to each student (the .unri t of anSaly$1s) > 
it makes more sense to us to use the My Xlass Inventory of 
Anderson and Walberg for this purpose. The MCI contains 45 • ^ 
items distributed over five scales:, Satisfaction; Friction* 
Competitiveness, Difficulty and Cohesiveness. It is Intended 

for use with^8^^2 year plds* who agree or disagree with eaxjh • 

, item. Although the Indiyidual scal^ reliabilities for Mfil 

are not so high as the more thoroughly developed Learning : . 

Environment Inventory (LEI) from which it is adapted 5 they 

are relatively high (.54 to .77)'ifor scales of this type used 
^ ' \. * ^ ^ 

with* children of this age group. Jt has be'en used successfully 

in a number of reseiarch studies and has been highly rated by 

several researchers. The instrument is Included in the. , 

Appendix* - 
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III. METHODOLOG Y \ 
. A. STUDY QUESTION ONE 

2. Sampling and Selection 

This section includes a discussion, of the sampling 
dimensions and sampling design for answering Study Question 
One. Also incltided are tested step-by-step procedures for 
selection and. screening. » 

a. Sampling Design 

i. Grade Selection - In order to increase 
the intensity of the study, the design is limited to a single 
grade level. The fourth grade is reccnmended for various reasons 
First, the inclusion of -grades beyond the fourth grade level 
would likely increase the complexity of the study by introducing 
another set of organizational arrangements such as different • 
school grade grouping and differentiated staffing patterns. 
Secondly, grades below the third grade level would prbvide (1) 
minimal variance on the criterion achievement measures, (2) 
minimal opportunity fqr individualization to operate effectively, 
(3) minimal testing opportunity since very young children are" not 
able t9„make the kinds, of distinctions called for on some of the 
affective measures, and (4) little opportunity to make use of cost 
saving devices such as machine scorable answ,er sheets. The final 
decision between the third arid fourth grades was coupled to our 
decision to recommend the inclusion of test exercises from the 
National Assessment of Educational Progress (NAEP). In order for 
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the study sample to coincide with one. of the NAEP sample groups 
the 9 year-old (fourth grade) group was selected. 

ii- ' Primary Sampling Dlfliensiarts- Study Question One 
inipl-ies that content area (reading and/or mathematics), instruc- 
tional type (individualized vs. standardized), and instructional 

• t 

settings (mainstream vs. separate" groups) be used as primary 
sampling dimensions. Since the major purpose of the study is ' 
the comparison of the effectiveness, of individualized, instruction 
with that, of standardized instruction, we regard the instructional 
type as the most basic of the sampling variables, tiext in im- 
portance is the Instructional setting variable. Finally^ although 
comparisons across content area are not specifically ea\ted for 
in the study question, our reason for a-sampllng design th*tMn- 
cludras all possible variations is to'^ investigate the possitrlltty 
of transfer effects in the situation where only math or only 
reading are taught in' one of the two instructional types. 
" , That is, there are four possible combinations of cqntent and 
instructional type. They are (1) both reading and math indiv- 
idualized^ (2) reading individualized and math standardized* 
(3) math individualized and reading standardized and (4) both 
standardized. In the eyent that all cells, described below, can - 
be filled the transfer of Qffect question can be investigated. 
(See below:} 
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Along the Instructional ■ typ e ■dimpws'tnn. hnth Individualized 
and standardized Instruction as well as various hybrid* types are 
expected to be Identified. For this study question no hybrid 
types will be included. 

•Instructio nal setting has only two categories; mainstream 
and separate Instruction. In attempting to define the categories 
, we had to infer the reason for lnterest in the instructional 
setting. This we took as stemming from a concern for the 
possible negative effects that are likely to derive from labeling 
a learner as slow or different. The effect Is believed to be . 
associated, with tracking (heterogeneous ability grouping) students 
Into low ability groups, and/or labeling, in effect, students 
as special or different if they are "pulled, out" "of class for 
.instruction. This interpretat1on,-calls^for a definition-of 
"mainstream" that describes conditions wherein labeling of any 
kind has been avoided and "separate instruction" as any condition 
^^at supports a "libeling" possibility. 

We anticipate that Instructional arrangements 1n the real 
world will make it difficult to differentiate "mainstream" from 
"separate instruction" and that math or reading instruction 
.following on the "pull ^out" condition wiU^me^l^e it difficuTt 
to categorize students as .individualized or standardized. The 
following diagram has teen constructed for purposes of definition. 

^ !7»— ■ , — 

♦Hybrid types have characteristics of both individualized and 
standardized instructions. See, III -58 for'a more complete 
definition. 
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We deftpe "mainstream" as the conditipn wherein students have 
been assigned to a class on a heterogeneous basis and are not 



pulled out for special instruction, 1:he |N.P.0.| boxes are,.thown 
above. Ideally, treatment Is mainstream only if it occurs in 
both content areas. All other conditions at the same level we 
define as "separate."- Siace "pull out" ihstruction'can be 
either basic or supplementary and can be either individualized 
or standardized there is likely to be a probilem in assigning 
students to the individualized or staiwlardi zed instructional 
type. .Because of this we iiave labeled those situations that we 
consider to be individualiied 0 and standardized © where the 
pull out instruction is supplementary. 

Thus, considering the combination of two content areas, 
two instructional types and two Instructiohal settings there are 
8 sampling points (cells) for a gra()e level as sh6wn in Table III.l, 



Table III.l . - 
Sampling Cells for Oi\t Grade Level 

Math Reading 
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i11. . Sample sl2.fi- In order to preserve >the 
intense routine of -the study, assure reliable informati'on, 
and comply with economic constraints, we have recommended 
eight classes per cell be selected. 

Unit of. Analysis . Since it is anticipated tbat the . 
students to be studied^ wi,ll be selected from "the class group; 
i.e., the Tast stage of a multi-stage sampling procedure, we 
feeJ justified in suggesting that an appropriate unit of 
analysis Is the student for Questions One a and One b. (Si-nce 
One c deals with degree of implementation of a progiHitn, the 
classroom is the obvious unit for this analysis.) 

Given the fixed budget and the N required for reliability, 
sampling at the class level would not be ' appropriate . Adequate - 
power for the statistical tests can only be obtained in this 
design by using a smaller unit than the classroom for Question 
One, parts a and b. 

1v, Control Variables- Variables to be 
controlled or partially controlled by selection are discussed in 
this section, ,Vaf\iables used for statistical control are dis- 
cussed fn the^rfialysis section. ' . . 

Obviously, pumerous variables are related to student outcomes 
in a variety of ways, it is patently. Impossible in an-intensive 
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study of this kind even to approximate an ideal experimental 
situation where adequate controls can be 'established or can" be 
assumed. Therefore, it is extraordinarily important that all 
possible effort be spent in establishing comparable groups at 
the sampling stage. District educational policy and policy 
implementation; school SES in terms of percent of Title I 
students, racial composition, and per pupil expenditure are 
considered as variables controlled by selection.' 

Since pairs of classes, one individualized and one stand- 
ardized, will be selected from a single district we will assume 
that district educational policy, and policy implementation' are 
satlsfactorally controlled^ In matching the class pairs, per- 
cent of Title I students, racial composition, per pupil txpend- 
Iture, andievel of urbanization must.be considered as primary 
control variables. At the student level, students for whom • 
English is a foreign language will be excluded frofli. the analysis. 
Other variables- that need to enter into the selection and 
matching decisions are discussed below under screening and selecti 
b.- Sci^ening and Selection Procedures - , • 
The intention of^e screening and selection procedures . 
is to identify for study purposes well -implemented 
individualized and standardized programs in readi ng and mathe- 
matics^ employed in compensatory situations at the elementary 
level , used in either mainstream or separate settings that have 
demonstrated replijcabil itv . 



Since this study is an intensive extension of the national 
NIE compensatory education study it Is also considered desirable 
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that the sample to be studied Include, sites identified for study 
in the District Survey I such "that they comprise approximately 
50X of the Survey II sample. ] 

•To meet these requirements adequately, it will be necess'ary 
to use multi-stage screening and selection procedures for the 
study. The steps to be employed in completing the screening 
and selection procedures are listed here and described more 
fully below. They are: (1) identificatipn of candidate indiv- 
idualized reading and/or mathonatics programs meeting the, 
replicability an.d grade criteria, (2) identification of field 
work. centers, (3) identification of compensatory individualized 
schools, (4) identification of comparable standardized schools, 
(5) screening sites on the implementation criteria, and (6)- 
final selection of schools and/or classes for study. 

Identification of Candidate Individualized Programs- There 
are several sources of info'rmation'that have the potential of 
identifying individual-i'zed programs for study. Initially a search 
of the professional literature jnd conmercial advertisements^! 
s'hould be made. For the foriner, there are several suggestions-* 
First, the.NIE has recently (1975) completed a computed* search . 
on'ERIC entries- in the area of compensatory education and indiv- 
idualized instruction 'in reading and mathematics^ Second, AIR . 
(Wargo et al.» 1971) has compiled two lists of descHp- 
tors from the Thesaurus of. ERIC Descriptors . 
(2nd ed.)» The first set contains 40 descriptors 
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"related to the' education of children , preschool thr^h se"h1oV 
high' school." A modtfled version of the AIR descriptors should 
identify some Individualized programs for the Initial candidate , 
pool. Finally, the NIE product catalog (In preparation), ALERT 
and EPIE will provide some Information regarding'indlvlduallzed . 
Instructional programs. 

For the commercial source, a telephone call to publishing ' 
houses advertising Individualized programs at the elementary 
level in reading and/or math should serve the dual purpose of 
adding to the list of candidate Individualized programs and 
securing nominees of sites qualifying as well -Implemented either 
at the district or school levels. 

Other potential sources. of Information Include non-profit 
organizations known to be developers/producers of Individualized 
programs, (e.g. , Wisconsin R&D Center at the University of 
Wisconsin, the LRDC at the University' of Pittsburgh, Research 
for Better Schools In Philadelphia, The Westlnghouse Learning 
Corporation In New York City). 

' Organizations and professional groups known to be concerned 
with compensatory education programs should also be contacted and 
asked to submit a .list of nominated IndlMlduallzed programs. Among 
those that might be contacted are: Compensatory Education Division 
Chiefs for the State Department of Education, State Title I Coord- 
inators, the Office of EconcHnlc Opportunity, U. S. Office of 
Education, the National Laboratory on Early Childhood Education, 
the National Education Association, and the American Council on 
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Education. 

Many of the sources specifted above will . upon request, 
submit a list of programs^ and sites that use-the Individualized 
programs. The site Information gained up to this point can be 
used to eliminate programs that do not meet the repllcablllty 
criteria, (I.e. , a proggam must be used In at least 100 class- 
rooms or In, two school districts). Programs also can be , 
rejected at this screening stage If the content area Is other 
than In reading or math and/or If the materials are not designed 
for the grade level called for In the study design. 

Identification of Candidate field Work Centers- As 
Information Is gathered Identifying Individualized sites, site 
lists should be compiled by subject area. 

Final lists -of schools using qualified Individualized 
programs should be prepared listing sites by program by subject 
area. That is, there should be lists of nominated sites: 

(1) for reading using program A, program B, 
program C, etc. 

(2) for math using program A, program B, 
program C, etc., and 

(3) for reading and math in a single site 
using program A, program B, program C, 
etc . 

Since the study will be an intensive one requiring extensive 
on-site observations 'to be mad^ over an extended period of time, 
it may be desirable to identify clusters of.sites around several 
centers from which coordination of the field activities will 
be controlled. The "cluster" notion is discussed fully in the 
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logtstfcaVsectton. If this course of action is foil owed, the 
sites Identified above should be plotted on a map preserving 
program and subject area Information. This visual aid should., 
facilitate thB selection of cjindddtte ffeld work centers, 
(See Figure III-l.) ^ ... • * ^ 
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Idfefttlfytnq CoBipfen$at6ry Stt6^; Sebo61 Dtstrlcts- At this, 
stage in the screentng procedure the three .Itsts contain the names 
of districts or of schools nominated as exemplary users of indiv- 
idualized math and reading programs at the appropriate grade 
level. The next step is to identify' those among them that 
qual Ify as - "compensatory . " 

There are two separate aspects to the "compensatdry" 
dimertsion that need attention. First, the student group reached 
should be "compensatory." Since Title I schools are Identified 
primarily- to serve this student group, it seems expeditious to 
select schools- from among this school- group. Secondly, the 
individualized program must be used with the "compensatory" group. 

It seems that an efficient selection procedure would screen, 
out, at this stage, districts rather than school sites. For 
example if a district is not located within the specified distance 
from a field work center, or if a district has- few compensatory ^ 
students, the district should be eliminated, school sttes need 
not be contacted. \\ 

It is reconmended that initial contact be made with the 
•district office in order ±o confirm the information regarding use, 
to determine whether or not sites are Title I schools, to detertnine 
whether or not there are other commercial or locally dejfeloped 
individualized programs in use in the district, and to request the 
nomination of comparable standardized sites for the individualized 
sites. CSee Appendix for District Telephone Interview Form.) 

Since it is most desirable to identify a set of , comparable 
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standardized schools wtthtrt the same district for each of the 
Individualized schools *1n order to control for variation In 
educational: policy and policy .implanentatioo » l^^mi nations. of 
'standardized schools should fallow Identification of schools 
using individualized programs. The school district perspn 
shoul^ be asked .to consider such criteria as similarity In 
terms of ethnic and, racial composition, student mobility, 
percent of compensatory students, and per pupil expenditure in . 
nominating a set of standardized schools for each of the indiv- 
idualized schools. (Sb€ Figure III. 2.). 

, It will be necessary to compTete screening pn the 
"compensatory" dimension, at the school level since the other 
aspect tp be considered pertains to the use of the programs with 
comp^ensatory students. . ' ' 

"Identification of Compensatory Individualized Schools- The 
screening process for individualized schools and for standard- 
ized schools", can be operated sinful taneously. For reasons of 
convenience, however, they, are discussed separately. 

At this stage, information regarding the number of Individ- 
ualized classes in the designated grade level, grouping practices 
instructionarsettings, should be obtained via a telephone 
' interview with the school principal. The telephone Interview 
should also be used to identify sites obviously not qualifying as 
either we-ll-implesnented individualized or well-Implemented • 
stahdardized. 
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The telephone interview fom for obtaining the information 
Identified above and for preliminary screening on' implementation 
is called the Telephone Interview Form for School Principals. 
(See Appendix) 

Identification of Standardized Schools- " The procedures and 
forms for screening, standardized schools are identical to those 
identified above. The major difference, is in the criteria to be 
employed iTj deciding whether or not the standardised program is 
well -implemented. 

It should be possible to eliminate at this stage schools not 
qualifying as either fully individualized or fully standardized. 

Screening Sites for Implementation- On' completion of the 
procedures described above, it will be necessary to visit the 
remaining candidate individualized schools and the set of schools 
' nominated* within the district, as potential comparison standard- > 
ized sites. The purpose of the visit. is specifically the selection 
of classes for study but at the same time additional information 
should be gathered for making the final decision regard.ing the 
comparison pairs. 

It should be recalled that the comparison of standardized " 
programs with individualized programs is only a comparison of 
different ways of organizing for instruction; In either the 
individualized or standardized case, it will be necessary to' 
eliminate from the candidate pool classrooms that do not have a 
desirable climate. Observations should.be made on the Screening 
device for classroom climate. (See Appendix for Classroom 

♦ 

Environment-Screening^ 
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Final screening for program implementation should also be 
completed at this time. We recommend that this, be done by means 
of an Interview with the^ teacher regarding her classroom practice 
vis-a-vis Ihe^ instructional tasks. ^Bee Teacher Interview Form- 
Screening) The criteria specified for the observation schedule 
(ITTOF) should'be used to eliminate programs not likely to qualify • 
as "well -implemented" as the term has been defined.* 
. - Other j'nformation at the school level that should be gathered 
foXconsi deration includes (1) likely cooperation that the 
school §,taff will give the study groups C2) likely level of. 
cooperation to be expected of the community and ^parent group* 
(3) like,1y conflict of study testing schedules with school, 
Title I, district, or state testing programs, (4) likely conflict ' 
with other ongtf1)ig^compensatory education studies. • 

Final Selection of Classes for Study- If a school has more 
than one class in the target grade level, such control variables 
as the proportion of Title I students^<ind racial composition should 
i^e considered in selecting study classes. Needless to say the 
instructional settings for each member of a pair of matched 
individualized and standardized classes should be the same. 

For any class selected it will be necessary to determine that 
there are at least five (5) students who qualify on both an educa- 
tional deficiency and a poverty criterion. (It is not clear which 
of the criteria is more relevant in t^rms of identifying the 
intended beneficiaries of compensatory funds. ) 



*See page'III-9. 
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Finally, the schools selected should be merged and matched 
with schools from the District Survey I. Classes should continue 
to be selected until 150X (96) of the minimum sample. size Is r«4ched, 
and the number of schools matched with District Survey I schools 
exceeds 75X of the minimum sample size. (See Figure 3.) 
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Figure .111.3. Flow- chart for schooV screening at school Tevel 
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III. METHODOLOGY - • 

A. STUDY QUESTI6N ONE .. 

: • 3. Data Coll ecti on Plan. ' . ./ ^ 

As the section on study variables and Instrumentation 
.implies, data wiVl .be collected for the study from various 
sources during the study period using many different data ' 
gathering methods. On the following page is shown a class- 
ification of alT the data gathering instruments for Study 
Question One. The study design calls for extensive^observ- 
ational data to be collected on classroom processes in 
addition to data to be callected by means of interviews, 
questionnaires, and tests achiinistration. 

a. Observational Data ,Col4ectiori 
Classroom observations for thU study question will be 
collected on the ITTOF to obtain an assessment of 
program implementation. The observations will be made by 
classroom observers, specially trained for the purposes of ^ 
the study and will, in general, be collected by a single 
observer recording data directly. on specially prepared 
optical scan sheets. Approximately 30^ of the observations 
will be made by a team of two observers for purposes of 
establishing the reliability of the data collected. In the 
cases of large open education classrooms, two. observers will 
be used to make observations. ^ 



iV. 



XZ 

o 
in 

t: 
o 



to 



s 

•of 



I 



CSJ 



CM 

03 



=9 





r— 












to 










































« 
























































































to 














s ^ >> 










O 
























0) E o 

•*-(/) UJ -fJ 










o 














U 0) z « 
























U) c 












A3 ^ W 








» 
















C ^ QJ v» 












U A) I. U) 












O C O) (O 












O O 
























f— 4J O- 
























































































C 






- 
























, 1 






2t 






«r 

s 












o 


















E 












tn C 












« o 






< 






t- li- 


















z 






^ c 












e »^ 
















< 








O "M 












T- « 
























































































































o 
































0) 




c c 










c 


c c w 








w 


ft 


4} (U o 








u 


■f*» 


0) OJ lU' — 








c 


4-> 


O O 7 








cn 


(J 


t/) t/) ^ 4) C/) 


















t 






OJ 4J ^ F 






UJ * 


(LI 




C QJ t 






o 






•r- OJ 4^0 






UJ 




O 


«' g C U- 






o 






C O 5 






UJ 


01 




O 4J OJ 










-P O 


•r- -r- O '1- 






< 


■5 


(Q O 


4-> > -r- > 

<n c u L. 








c 


(— J f ^ 


OJ UJ 4J OJ 












3 • l/> 4J 








o 

43 '-"^ 


-P 


O E -r- C 

O ""O 








i — h- 




. W. C/) O CO 








1 ^ 




QJ 1 t- * 1 ^ r— 








(U 


"O o 


43 CJt iA UJ O O 








cn J 




O t/> O O O 












10**-^ <0*«— '43 43 








o 


-P 


•» f- O O 










00 


h- O to (/) 




- 






•It 


« « « « 


















ui 












o 












a: « 




u 








z> 


OJ 




i 






o 


c 










CO 


u 


O 




m 






to 


<a 












o 


s 




u o 






1— 


a. 


o 


CO o 



ERIC 



III-32 
59 



r 

The program implementation observers win collect data 
during those parts of the day designated as being the primary 
time for Instruction In the basic sjdlls area. Observations 
for program Implenentatlon will be made over the entire school 
year so that .it should be possible to Identify any deterioration 
implementation that may occur. During- the observation V5;^ts, 
the trained classroom observers will use the ITTOF ta record " 
data reflecting dirfect observations of the Instructional 
process, examination of various instructional materials in- 
^cluding work completed by students and records kept regarding 
the type of instructionail tasks observed. The observers will • 
also examine student i'nstructional materials prior to, during 
and following the class. These observations will focus on 
the^ scope of the curriculum opportunity and the specificity 
of the instructional objectives of the teacher. Informal 
questions asked of teachers and students will also enable 
judgment to be made about how. each is attending to the in- 
•structionaT tasks during the specific class period observed. 

All observational data will be forwarded on a weekly. 

basis by the observers to the tzentral facility of the con- 

traetor for appropriate processing. 

b. Test Data Collection 
♦ 

All standardized tests such as Lorge-Thorndike Intelligence 
Test (LTIT), 'and California Achievement Test (CAT) as well as ' 
My Class- Inventory (MCI) will be administered under standard , 

_ ^ -A 

conVlitions. LTIT will be administered i'n"tl!e^n as a pretest. 
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CAT reading and mathematics tests will be given to the ■ 
students In the study group in the fall and spring on a 
pre- and posttest basis. Only posttest data will be analysed 
under Question One. National Assessment of Educational, 
Progress (NAEP) exercises which '>are criterion-referenced tests 

' 0 

and MCI will be given in the spring only. 

All tests will be administered by trained test admin- 
istrators in the presence of classroom teachers in order ^ 
to control test administration conditions. 

Specially designed optical scanning answer sheets will 
be used to reduce coding and keypunching errors and to 
fa oil Hate the data processing. 
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III. METHODOLOGY 

A. STUDY QUESTION ONE 
. 4. Data Analysis Plans 

In this section we describe all tasks and procedures 
that are involved in data analysis from the first stage of 
receiving data frcTi the field to the last stage of producing 
(printing-out) the final analysis results.^ A general outline 
of these procedures is shown below, and step-by-step 
procedures are discussed fn subsequent sections. 




Figure III. 4 Flow Chart for Datk Management, Data Procesaing, 

and Data Analysis 

a. Data Management 
Data management procedures followed for this study include, 
check in, editing and qual.ity control, coding, optical scanning-and 
data cT^taning; all designed to improve the accuracy of the data. 
Data classification,, record keeping and filing systems will need 
to be maintainc<j so that th« data files can b« continuously updated 
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and that Information which ra^iy be naeded by the prlncipa.1 
Investigator or by other research personnel can be readily 
retrieved at any st<ge of -processing operation. 

i. Check-in — The check-in system would 
operate so as to monitor the receipt of data from the field 
so that close csntrol may be kept over both individual 
interviewer's workflow and overall study milestones. Trained 
clerks will check in all data sets, verify that data are 
identifiable, contain complete Identification information, 
check for completeness, classify the data, assign data set 
codes, and keep daily and weekly tallies required for completion 
reports . 

Because of the size and complexity of this study, a 
computerized logging-in system may be used to supplement check-i 
procedures. Here, essential call report informationi would be 
keypunched daily and linked in the computer with a file con- 
taining the total sample. Weekly completion reports may then 
be generated with^ breakdowns needed for assessing study progress 
within particular sample sub-populations. 

•ii.- Editing and Quafity Control - After data are 
checked-in, every completed form will need to be reviewed by an 
editor who is supervised by a quality control staff. Editors 
should be trained in both classroom observation and testing 
procedures, and should attend a training session for this study. 

• The quality control staff will work closely with other 
♦research departments and early in the f1 el dwork' period provide 
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'feedback about any problems wtth the Interview or observation 
schedules that may require clarification of instructions. 
During this same period, the first interview conducted by 
each Interviewer will need to be checked carefully, the quality 
of this first interview should be approved by the fieldwork 
supervisor before the interviewer is permitted to continue 
with a field assignment. 

In reviewing a completed questionnaire, the editor may 
find that certain responses need clarification or that the 
Interviewer has misunderstood instructions or has omitted 
essential questions. In such cases, an explanatory memo 
would be sent to the-fteldwork supervisor and to the inter- 
viewer with copies of the questionnaire pages where the 
problem occurred. When necessary, 'the interviewer would be 
instructed to contact appropriate school personnel to obtain 
the missing information. . , 

iii. Data Classification System- Since data 
for this study will be collected from many sources the levels 
of data reduction are expected to be different- for different 
types of data and for different purposes. 

For convenience, we propose that a two-dimensional lay- 
out for data classification be used: (1) data source and 
(2) time of the data collection. The data sources should be 
broken doWn to (a). student level, (b) class level, and (c) 
community level. Data- will be also classified into (a) ante- 
cedents, (b) transactional , and (c) outcomes data according 
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to the time the data were collected and/or intended us^ of 
the data. The data that would be collected for this study 
question are tentatively classified into such a two-dimensional 
classification sj/ston. (See Table IIIJ.) 

iv. Data File Management- With various types . 
of data from many sources arriving at different times, 
there will be a jieed for a good data filing system. This 
system should allow for easy, efficient storage and retrieval 
of raw data via computer. 

On a weekly basis the J'new" data should be merged with 
^the "old.'' .All data-- both new and old-j- should be partitioned 
and stored according to the classification scheme indicated 
above. All data will be stored on jtiagnetic tape with disks 
used during the merging process. 

The merging process will involve: (1) writing each old 
data set from tape to disk, (2) adding- the new data to their 
respective data sets, and (3) writing each data set back to 
tape from disk. All of this wilVbe completed by a user-, 
written computer program that flags the beginning and ending 
dt^sk tracks off each data set. 

b. Data Reduction and Transformation 

The leve,l of data reduction will vary with the source of 
data and with the unit of analysis for the study question. 
Since the unit of analysis for Study Question One is a com- 
pensatory education student, th'ese students' outcome data should 
be scored for each student when the data files are completed. 
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This design has recoranended the use of the California - 
Achievement Test- Reading (CAT-R» posttest). California 
Achievement Test- Matheuatics' (CAT-M, posttest). National 
Assessment of Educational Progress Test In Reading (NAEP-R) 
and NAEP test In Mathem'atlcs (NAEP-M). and the My Class 
Inventory (MCI) as student outcome measures In answering Study 
Question One. , 

Student files- Data should be stored In three separate files, 
by student. The three files should contain data for students 
meeting (1) only the educational deprivation ertterlon, C2) only 
the poverty criterion and, (3) both the- educational and poverty 
criteria.- The student data filed will be individual SES, I. Q., 
test scores, and Infomatlon Identifying the sampling cell to 

which the student belongs. 

^ All files will need to be checked for completeness with 
regard to the criteriori in question. That is^ for students 
meeting only the^education deprivation criterion and students 
meeting both criteria the contractor wiM need to check for I..Q. 
data. The absence of I. Q. data in thisjnstance should eliminate 
that student from the analysis. For students meeting only the ^ 
poverty crittri*Dn and students roeetirig both criteria a check will 
need to be^made for SES data. Students" will be dropped in this 
instance tf SES data are missing. Students whose data are in the 
thtrd ftle should be those who qualify on both criteria » SES and 
IQ. • ' 
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Scoring- Students' achievemen^test item data on the 
Caltfornia Achievement Test (CAT),^rid National Assessment of 
Educational Progress exercises CNAEP), and intelligence test 
data at this stage has betn stored on tape from the optical 
scanning process. The student's CAT subtest and total raw 
score!; will be converted into Achievement Development Scate 
Scores (ADSS) by using national norm tables. A subroutine 
program for converting CAT raw scores .to^ADSS should be used. 

When the criterion-referenced exercises (National Assess-" 
ment of Educational Progress exercises) are scored, they will 
be classified into appropriate size and community groups 
according to the NAEP classification sch-fene. The item analysis 
procedure used should provide for each exercise the proportion of 
students who selected the correct answer(s). It is thfls._proportii 
that will be used in the comparison stage. ^' - 

The measure of classroom climate (My Class Inventory) will 
be scored for each of the five scales by sunming positive^- ' 
responses. 

■The Lorge-Thorndike item data will be converted by using 
norm tables. A subroutine will be needed for this purpose.... 

The degree of implementation data that is needed to answer 
Study Question One part c will be derived from the data collected 
Instructional Task Treatment Observation Form (ITTOF) during the 
study period in the following way: (1) assign value 1 for each " 
instructional task treatment practice if it is consistent with 
the definition (that is, if the instructional event occurred in • 

/ 
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an tndt vidua Mzed way. In Individualized classes and Irra 
standardized way In standardized classes), and otherwise ass-ign 
0; (2) add these points icross the ten essential Instructional -<t 
^task treatments 'and across the entire series of observations ; 
and (3) convert the sum Into a percent. This data reduction 
procedure will be easily handled with the TRANSFORMATION 'program 
In SOUPAC (Computing Services Offices of University of Illinois, 1974) 

For example, If there were 5 essential task treatments and 
6 observations were made in' an individualized classroom the- 
observational data might look like those shown below in the 
following tables: 



Observation 1 Observation 2 . . . .Observation 6 




4 points + 5 points + +4 points = -sum (27) 

■* Obtained Sum ....... 

Percent = X 100 

* Possible maximum sum 



The decree of Implementation index for this class will be 90., 
27 X 100 = 90%. 
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In the tevent that deterloratiion is very high over th^ course 
of the year 1t may be necessary to establish cutting point such 
that an adequate number of sites are retained for the primary 
effectiveness comparisons. The contractor will have to decide on 
a cutting point for identifying well -implemented programs for Study 
Question One part a and b. What the cutting point should be cannot 
be specified at this time..s,itnc6 there is no way of knowing what 
the distribution of implementation scores will be, 
c. Analytic Procedures 

In answering Study Question "One, we recommehd that analyses 

be* performed based on three different criteria for identifying 

compensatory education students: (1) a poverty criterion (e-g-, SES), 

(2) educational deficit criteria (e-g-, educational achievement), 
f ' 

and (3) both poverty and educational deficit criteria. The 
following analytical procedures for question one will be employed 
for all three grotip^ identified by these three criteria. 

First, there is a need to consider sampling bias prior to 

r 

analyses. Matched samples are often used to reduce bias in non- 
'randomized studies such as the present one- Unfortunately, it 
Is Vikely to be difficult to obtain satisfactorily matched samples. 
Aa alternative' is to make statistical adjustments after the 
iamples ha'Ve been drawn. We would recommend -that the uncontrolled 
.effieets of confounding variables s.uch as SES, I.Q.; etc., be * 
partial led out by treating confounding variables as covarlates in 
•an.^ analysis of cbvar lance design. 

] In-order to investigate the relative effectiveness of indi- 
vidualized vs standardized ijistruction and mainstreamed vs separate 
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instructional settings, we recommend a 2{progrdm) x 2{setting) 
factorial design employing univariate and multivariate analyses 
of covariance on reading, mathematics and student perceived 
classroom climate. It is possible that there will be program 
transfer effects either carrying over from reading to math or 
from' math to reading. Four different instructional groups may 
be easily identified as (1) individualized^ instruction for 
both reading and math, (2) individualized instruction for 
reading but standardized for math, J^3) standardized for reading 
but individualized for math, and. (4) standardized for both 
r^ading^ and math. We recommend that transfer effects be 
investigated using the same 4{combinatipn of instructional type 
and content) x 2{ instructional setting) ana'^tic scheme* 

Sirtce we believe that program effects are cumulative, 
posttesf achievement scores instead of residual gain scores, 
should be employed' as the dependent variables in this analysis. 

We recommend that univariate analyses be performed on both 
the CAT subtest scores and total scores of reading and math^ 
and NAEP reading and math total scores, and student perceived 
classroom climate (MCI) facTtor scores separately* Multivariate 
analyses should be run for; (1) the four CAT subtest scores 
(vocabulary, comprehension, computation, and concepts and 
problems), (2) CAT reading total scores, CAT math- total scores 
and student perceived classroom climate, and (3) NAEP reading 
and math total scores and student perceived classroom 
climate. In the univariate cases, a priori planned multiple 

* 
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comparisons or Duncan's multiple' range comparisons need to be 
performed for those contrasts showtng significant differences; 
while fn the* multivariate cases, discriminant analyses' will heed 
to follow those contrasts showing, s'ignificanf differences. ' 

In. order to investigate the relationship between instruc- 
tional types and student outcomes and between instructional 
settings and' student outcomes, Hay's (1973) OMEGA square (W^) 
will be computed for univariate cases,' and Tatsuoka's (1971) 
multivariate version of OMEGA square for multivariate analyses. 

Either MANovA (Multivariate Analysis of Variance) or 
MULTIVARIANCE (Univariate and Multivariate Analysis of variance, 
covariance and regression), or SOUPAC (Statistical Oriented 
Users Programming and Cons^ulting) can be used for these analyses. 
One hypothetical table is shown as follows: 



.Univariate Ahal-ysis of Covariance Table 



Source ^ 


^ss . 


DF ' 


'MS 


F 


Programs 


444.0 


3 


148.0 ' 


5.67 , 


Settings 


' 146.0 " 

. C ' 


> 1 


■ 146.0 


,'"5.59 V 


Programs x' 
Settings 


24.0 


' 3 


• 8.0 


0.31 


Erhor 

1^ 


'939:6.' 


,36 ' 


26,1 - 
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DISCRIMINANT ANALYSIS 
Mean Scores and Standard Deviations 

C^roup.X (M) Group 2 (S) 





Mean 


SD ' Meaii- 


.SD 


Var. 








VCBL 


25.30 


2.80 .20.20 


2.90 


CMHS 


21.30 


3. '40, 18.90 


3.10 




30.25 


5.01 25.51 


\ '4.50 


CPPM 


27.38 , 


4.85 23.48 


4.30 


Function 


1 ' 






Coefficient . 


Standardized Coefficient 


VCBL 


1.25 


-VCBL 1 


0.35 


CMHS - 


, 0.89 . 


gMHS 2 


. 0.12 • 


CMPT 


3.80 


CMPT 3 


' 0.84 ' ■■• . 


CPPM . 


4.50 


■ CPPM 4 


0^72 


Constant 


;.4.57 






« * * * ' 

If the cont>*actor^has serious reservations about • 



use of • the analysis of covariance we recoirmend regression 
adjustment (Cochran and Rubin, 1974) as an alternative 



' i'B'tthich the treataiient effects are adjusted by th^ 
. regression of the dependent variable on the confounding 
.variables. In this' case, Hotelling's T^'and Student's 
t-test would be used 'to test the adjusted treatment 
effects- standardized vs indivtaualized and mainstreamed 
vs, separate instruction for multivariate and univariate 
analysis respectively. ' 
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' To answer Question One part c, th^ outcome data o? 
students .in not well -implemented classes will, be restored. 

The relation between student Outcome variables 
and the -degree of program implementation will be investigated* 
via a correlation approach/ It would be desirable initially 
to determine whether any of the relationships areVnon- linear. 
,The analysis of Variance, test for non-linearity (H. M. Blalock, 
1960)'can be employed to *det'ermine the proportion of variance 
explained by the linear model and to test the significance 

T ^ 

of additional amounts of variance explained by the^non- 
linear mode].- If there is a non-linear relationship between 
achievement raeasures and the degree of implementation 
.yaHable, some transformations (e.g., square .root, / 
logarithmic^ etc.) should be made prior to analysis. Pearson's 
product moment correlation coefficients between students' 
achievement scores and degree of program implementation ihdi'ces 
will be cpmpQted for -individualized and standardized instruc- 

■• ' • . *• • • 

tional groups, separa.tely. - 
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III. HnHODOLOGY ' " 

B. ' study question two 

"I- Rationale for Selection of Variables, Definitions' 
and Instrumentation ' . 

We are not confident that direct comparison of groups of 
classrooms labeled individualized and standardized will disclose 
'any significant differences -on achievement or climate outcomes 
because of. confounding of treatments. -Thus, we recommend 
investigation of the differential effects of differential 
treatments of instructional tasks across all classrooms. This 
does not mean that individualization and standardization are 
to be Ignored; rather, we recommend study of the conjoint effect 
upon outcomes' of the degree of indlvidualizition* and lisvel of 
attention given to the instructional 'tasks by classroom without 
.regard to any global classification of that classroom". 

Moreover, since it has-been argued that classroom climate 
inter;acts with instructional treatments with consequent impact 
upon cognitive and affective outcomes, we reqommend that the • 
nature of - these relationships be studied. ' 

♦ » 

Based upon the concerns noted above, we reconniend the 

second set of major study questions. 

■> . ' ' , 

a* How doee apecifioity of. treatment of > 
the^.imtruotional taaka-y taken one at' 
a time and in , aec^ential oombinatiom^ 
relate to oognitvoe and affective , 
outoomea? 

■ ■ 
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b. Row doea apecifiaity of treatment of 
the instructional tqfiks taken together 
with olaaaroom climate relate to 
cognitive and affective outaomee? 



• The variables of major concern under this set of 
questions are: 

Program Variables 
Classroom ,Climate 
Reading' Achievement 
- Mathematics Achievement 
Student Perceived Clim&te , 
Student Attitude Toward Reading 
Student Attitude Toward Math 

* < 

Program variables under this question consist of the 
same ten instructional tasks discussefd under question one. 
However, while the tasks varijsd in terms of instructional unit 
size under question one, tas'k treatments under question two, 
will vary in terms of the specificity of their treatment which 
is (Refined in term's of coordinate descriptions that reflect 
(1) instructional unit size and (2) level of attention given ^ 

to the task. 

The reason for this btdimensional concept of task ' 
treatment derives from concern that differential treatments 
of instructional tasks within individualized and standardized 
groups^ have washed out potentially significant differences 
when outcome, effects are aggregated for program types. If/ 
significantly different outcome effects are to be found, they 
will be associated with instructional practices in the^ classroom 
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without regard for labels. 

^While educators are ambivalent with respect to whether 
or not individual students can.te best atteryded individually 
or -as a member jof a group, as was mentioned under question one, 
they ire equally divided on the level -of attention given to 
instructional tasks that will most likely lead to highest 
achievement. One group tends to be determinist; i.e., they 
believe that, given the- antecedent c»nditions, instructional 
sequences can be specifically planned and arranged so as to 
lead the student to predi.cted outcomes/ Instructional task 
treatments in this case tend to be detailed and structured, 
i.e., highly specific . 

Another group of educators tends to be phenomenal Ist, 
given to the veiw that learning is a unique experience in 
time and space and a function of the stage of development 
of the learner. This group is committed to open and relatively 
unspecified learning environments. They count upon a natural 
propensity of humans to learn according to their res'pective 
interests, -needs, stages of -development and circumstances. 
Between the two groups are those who pursue eclectic mixtures 
of instructional task -treatments employed by the detefminists 
and the phenomenal ists. . . - - ' 

To generalize an answer about "the relative effectiveness 
of Individualized instruction vs standardized instruction" . 
would be meaningless considering all the permutations and 
combinations of organizational arrangements and instructional 
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t^ask treatments that would be subsumed under the labels > 
Individualized and standardized* We perceive one way of'* 
categorizing the variety of instructional conditions with 
res>ect to task treatment to be as follows: 

Instructional Unit Size 

Level of Attention Individual Variable Class 

Subgroups Group 

m 

Specific 9 . 8 

Eclectic 7 6 

Unspecified 5 4 



The ordinal numbers appeatvlng in the cells indicate an 
order of specificity of attentio^ to the instructional tasks , 
from least (1) to most (9). Such matrices can be used to 
describe the specificity conditions for each of the bi- 
dimensional instructional tasks. 

Provision of Curriculum Opportunity- Appropriate 

matches of learners with instruction are dependent, in part, 

upon a curriculum that permits initial placement of each 

learner on the basis of his/her entering achievement level 

and provides sufficient scope to accommodate the potential 

progress of#the most advanced learner. The following matrix 

describes the specificity conditions for the opportunity task. 

t 

It indicates that the x^portunity that will most specifically 
accommodate the unique placement and progress needs of 
learners occurs when the scope is three-or-more grade levels 
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' 3 
2 

V 1 



and placement is to be made for learners individually. 



Level of Attention 



CURRICULUM OPfORTUNITY 
Opportunity Is provided for: 



Individual 
Learners 



Variable Class 
Subgroups Group 



3 or more grades 
2 grades 
1 grade 



a 

7 
5 



8 
6 
4 



3 
2 
1 



Statements of Curriculum Intentions- Deliberate matching 
of learners with instruction and evaluation of th,eif per- 
formance can be most specifically attended wh^r^ sta^tements of 
curricular intention are expressed tn terms of cohl^ent 
behaviors expected of learners individually. 

Deliberate matching of groups of learners with instruc- 
tion and serendipitous matchings of Individual learners with 
instruction according to their discovered needs or interests 
may be best accommodated when curricular intentions are 
expressed as broad goal statements » relatively unspecified. 



ERLC 



Level of Attention 



STATEMENT OF CURRICULUM INTENTIONS 
For: 

Individual Subgroups Class 
Learners Groups 



Pupil Content Behaviors 
Content Offerings 
Broad Goal Statements 



9 

7 
5 
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8 
6 
4 



3 
2 
1 



CurHculum Placement Decision- Instruction likely will 
,be more effective the more appropriately the learner is placed 



and need. Since the appropriateness of pTacement relative , 
to the learner's actual competehcy and need is a very high 



ness of placement be judged upon the basis of the placement 
decision conditions. 

Deliberate placement of learners will be most specifically 
accommodated for individual students to the extent that theiV 
competencies and needs can be described in terms of their 
criteriofi-refpre,nced performances. Less specific judgments 
of teachers and learners provide more latitude for learners 
to find their own entry points into the curriculum. Delib- 
erate placement of* groups on the basis of grade level expect- 
ations are probably least, specif ic with respect to the 
competencies and needs of the learner. , , 



into , the curriculum. an the basis of the learner's c 



ompetency 



inference tasK for observers, we propose that the appropri^be^- 



PLACEMENT DECISION CONDITIONS 
For: 




Individual Subgroup Class 
tearner Group 




Cri teri on-Referenced 
Performances 

Judgment 

Grade Level Expectations 



7 
5 



6 
4 



8 



3 
2 



III-52 



79 



Adjusting Rate- of Instruction- Rate of instruction . 
should be directly related to the "rate of acquisition of the 
jlearners whe^er ^ controlled , by the teacher or thejearner. 
^ince learners differ-RWirkedly in their rate of-acquisltion, the 
rate likely will be more specifically appropriate to the 
individual the smaller the group for which it is adjusted. 

Individual Subgroups Class 
Learners . Groups 

Rate of Instruction 3 2 1 

Provision for Individual Responding- Opportunities 
for responding can be provided for groups and can vary 
widely in terms of the degree to which the opportunity can 
be made specifically appropriate to- the needs of the learner. 
However, response opportunities are typically made available' 
to individuals, everf i^n group-paced contexts, and since 
judgments' of appropriateness mdst be yery high inferences, 
we recommend frequency of .occurrence be the dimension of 
variation for this task. If two-thirds or more of the class 
are provided response Opportunities- ih any given session this 
should be considered high frequency, .one^ third to two-thirds' 
would be cons tdp red moderate; and one- third or less low. 

Provision for Individual Feedback- Feedback, as with 
responding, is most often made available for individual 
assistance and also requires higli"inference judgments as to- 
its appropriateness for the individual , therefore we recommend^ . 
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the same frequency measure as for responding. 

Monitoring Individual Progress- Monitoring of .indiv- 
idual student progress can take many forms and is difficult to 
judge. We recommend' the same frequency measures as for 
the previous two tasks. 

Performance Standard for Advancement- Standards wi-ll 
more likely lead*to' performance consistent with cur/icular 
intentions when the standard is specific to the intentions, 
absolute for each learner. Variable standards fo/ indiv- 
iduals and groups will be less specific wit^^/pect to 
Intentions and no standards are apt to apply when only 
broad goal statements exist. 

PERFORMANCE STANDARD FOR ADVANCEMENT 
For:' 

Level of -Attention Individual Subgroups Class 

Learners Groups 

Absolute 9 '8 3 

Variable 7 6 2 

No ^ 5-4 1 

Evaluation of Performance- Program prepared tests are 
apt to provide the most specific basis upon which to judge 
performance relative to curriculum expectations, and the 
evaluation will be most specific for the individual the ' 
smaller the group being evaluated. Judgment of teachers and/or 
learners likely will be least specific with respect to the 
curriculum and large- group evaluations least specific with 
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respect to the .performnce of individuals. 

Matching Learners with Next Instruction- The reasoning 
associated with the matchtng of Ifearnersjsti^instructlon is 
nearly identical with that for the placement decision. 
Criterion-referenced performances provide the most specific 
information WItK respect to learner competencies and needs 
and automatic matching of learners with next instruction 
in the standard sequence is l.east specific relative to 
learner needs. The smaller the group being matched with 
the same instruction the more likely the matching will be. - 
specifically appropriate for the individual learner. - 

Clas| | pom Climate Variables- , Question Two requires an 

extensive study of classroom climates and -the relationship 

' ■ ■ < /' 

of various climates with changes In class achievement that J 

will occur during the period. of the school year. Soar and / 

■Soar have done notable work, in this area and have <ieveloped 

a set of Instruments that have proved effective and reliable , 

In discriminating among climates. 

The instruments proposed for the -measurement of classroom 

environment are the Florida Climate and Control System (FLACCS) 

(Soar and Soar, 1973)\and the Teacher Practices Observation 

Record (TPOR) (Brown, 19S8). In earlier work In Follow Thriough, 

both instruments demonstravted adequate reliability, and dis- 

cr1n)1nated at statistically significant levels between programs 

1n t|je Follow Through Planned'Variation studies. 
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*■ Sthce the' proposed study wtll exjunlne Individualization 
vs. standarciizatlon and- mainstream vs separate instruction, 
''^t seems likely that a different set of factors would emerge 
•from analysis, so .that a specific set of factors is not 
^^roposed here but will be derived. The kinds of behaviors 
'.whlch-arg recorded by the two Instruments are as follows: 

^ ' FUCCs, Teacher control of pupil behavior^ ranging from 
gentle and unobtrusive to harsh and coercive, both verbal 

f.and. non-verbal ; pupil response to teacher control (disruptive 
^behavior, orderliness and a global measure of task involvement); 

! pupil assumption of responsibility; teacher affect expression 
(positive, and negative, verbal and non-verbal); pupil affect 
expression (positive anci* negative, verbal and non-verbal); 
classroom structure (number and size of groups, with or 
without an,.ddult); degree of attention by adults to ind.ividua-l 
pupils; and pupil freedom of movement. 

lEOR- Is the teacher the center. of attention or is the 
pupit'^ is. the pupil actively involved, or wa^yting, watching, 
and \istenihg; is the pupil at wor4c on a teacher-set. task or 
his flWn; Js the subject matter ^iiask cleairly focused and 

^ restricted Otis thepupil encouraged to go beyond this; is 

/only^ one answer acceptable or aVe alternatives accepted; does 
the pUpiV support hi^? answer with evidence; who evaluates; 

^an'd Is motivation extrinsic or intrinsic? 

, ^ ^utcome Variables- Since an interest of Question Two, 
fotusesf'upon the relationship of climate factors and instructional 

'/ • ILI-56 

83 t 



task treatments wtth cognitive outcomes, the CAT. selected to be 
administered as a posttest for QuesWon One, will also be 
administered as a pretest here in order to provide the 
necessary gain scores for the analysis. The CAT was discussed 
under Question One. , r'^X^ 

A second Interest under question Two is the relationship 
of climate and Instructional task treatments upon non-cognitive 
outcome^. As with question one, we recommend My Class 
Inventory to assess student perceptions of classroom climate. 
The MCI Was also discussed under Question One. Al^g with 
student perceptions^ of classroom climate, it is of 1n*tr*st 
to_^now how the students' attiUide towards reading and math 
relates, to various classroom climates and instructional,/^ . 
treatments. The How I Feel instruments have been selected for 
th^s purpose. - . / 

• HIFAR and HIFAM are designed for the measure of student 
attitudes toward reading and math, respectively. Continuous, 
use'&nd revision of these instruments for formative and 
suiTinative.eValuatipns of RBS' individualized' Instructional 
programs have 'shown that these instruments cah he readily 
administered to fourth grade students." Internal consistency, 
coefficients for recen^tly revised versions of these measures 
were all above .80. 
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III. METHODOLOGY . ' ' ' 

B. STUDY QUESTION TWO ' . , ' • 
2. Sampltng and Selection 

r 

a. . Sampltng Design ' , ' . ^ 

An of the primary sampling ^dimensions' for Study' Ques^t ion' 
Two are the , same as those for Study Question One (See Sec'tion 
III*'A.2.) with one exception, an additiona.1 level of instruc- < 
tional type that we wiTl call 'the hybrid case. The hybrid 
case is defined as a classroom operating with 'mixed character- 
istics on the set of instructional- tasks as they have been ' 
defined for identifying individualized amd standardized ' 
operating programs. 

Inclusion of the hybrid cases as an instructional typef 
will increase the number of levels of combinations of 'instruc- 
tional type with content by five. They a,re: 

(1) both reading and math are hybrid, ,. 

(2) reading Is individualized - math is hybrid,. 

(3) reading is a hybrid - /nath is individualized, 

(4) reading Is s(bandardized - math is a hybrid, and 

(5) readiVig is' a hybrid - m4th is standardized^ 

This will increase the total number of sampling points^'" 
by ten with 4 classes per cell. This Increases the total 
sample size by 40 classes thus the total number of classes 
^ 10^. (See Table III. 3.) j- 

Unit oHanalysIs- For Study Question Two we propose to use 
th^, class as the unit of analysis since we-are studying the 

effecfs of process variables on the student group as a whole. 

/ 
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Table 111,3 

Additional CUsses to be Sarr^led for;' 
SQ 2 i.SQ 3 





6K 


R-I, M-H 


R-H* M-I 




r-h; h-s 


M 


4 




4 


\ -I 

4 


. 4^ 






4. 




4 


4 

•* 


i 




8 


8 


. ,8 • 


8 ' 



• • 3. i^ata Collection Plan ' " , . . 

The data- collection plans for the- major portion of t^e data have- 

been describedTabove (see Section Iir A.'3.) with a single exception. 

For Study Question Two we have recomnertded use of the SOAR observational 

systems for classroom climate. Xhe speciaV conditVons under which the 

proposecl' observations wou]d be made require consideration of a number . 

of alternatives particular to the observations required for assessing 

classroom climate that are discussed below. 

Rationale for Making the Full/School. Day the Unit of Observation 
for Classroom Environment ■ It is clear that observation of the 

instruction of reading and math would be required as a par^'^ bf the 

research design in order "to test the effects of program and setting on 

> 

classroom environment. Whether the research design requires observation 
of more than reading and math periods js less clear. For a number of 
reasons, we propose observing for the entire. school day, keeping separate 
't,he observations for instruction during reading and math, this woyld 
contribute to the., strength of .the study in several ways. First, it would 
provide Information about the integration of the compensatory program 
with non-compensatory activities (for example, does the presence of 
individualization for one subject matter. increase or decrease, time spent 
on other instructional activities?). It would provide .information on the 
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broader effects of individualization on the classroom'environment (for 
example, if individualization of reading is ass'ociated with a warmer * 
emotional climate, does that increased warmth carry over to the re- 
maindjer of the school day?). 

' Beyond these reasons for observtn^TThe full day, there is the 
broader question of whether learning in\eading, for, example, is. only 
effected by the activities that occur during reading instruction. 
Conventional wi'Sdom -suggests that this is not true. A teacher may, 
and probably does for example, teach reading during social studies or , 
other activities. Further, it seems likely that the effects of en- 
vironmental variables present dOring the remainder of the day would 
carry over to reading instruction. The pupil 's experience for the 
entire day probably affects his liking for school and the teacher, and - 
these attitudes in turn probably modify Specific subject-matter achieve- 
ment. 

In addition to the beliefs cited'above, there is, some evidence 
fdr collecting observation data throughout the day. Several studies 
indicate that pupil- achievement gain is greatest in classrooms where 
an intermediate proportion of activitiesasre set by- the teacher and that 
there is-less pupil gairt in classrooms where there is either a ^greater 
or lesser proportion of teacher-assigned activities. - (Soar, 1968; 
Coates, 1970; Soar and Soar, 1972, 1973; Broflhy and Evertson, I9V4). 
If this is so, then il; seems possible that the effect of whatever^^ amount 
of teacher task assignment may occur under individualized instruct^ion 

may be ntoderated by the amount of teacher assignment of task which joccurs 

j 

during the remainder of the' school day. ; 
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A final conslderdtton In .this reconiDendatlon is that 
collecting .obstervational data for the full day would be little' ' 
more expensive than, observing only durtng the time set aside for 
• tnstructton of reading and math, stnce teaching of reading and 
math wtll be concentrated In the morntng hours in most class- ^ 
. rooms-. Consequently; observing only those subject-matters 
wpuld make poor us&,of observer time. Full -day observation 
would ea^e difficult'scheduling problems. • • 

Rationale for the Proposed Data Collection Procedures - A review 
of the instructional patterns, likely to be encountered in classrooms^'in 
the proposed project indicates that they are much more complex and 
variable than has been true in past research utilizing classroom ob- 
servatlon. The norm in past studies has been to use eithey- -seli^-contained 
classrooms in Elementary schocT, or a s-ingle subject-matter or even a 
single brief unit of instruction in secondary school. In all cases,, a 
given groug of pupils was associated with a single adult or team of ' 
adults during the period of . the study. In contrast, the classrooms 
to be studied in tHis project represent varied and complex instructional 
patterns in which different teachers are responsible for varying portions 
of the total instructional time for a group of pupils. An adequate 
degree of reliability of the observational measures is., sought for each 
portion of the day for which a different teacher is responsible, but the 
data still must be aggregated to represent the' instructional experience, 
of ^the typical'chitartfotti f^or reading and math, and for the entire 
day. This complexity presents formidable problems in obtaining adequate 
reliability, only one of Which is the geographic separation of the sites 

1 % 
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which makes additional visits to. a classroom expefis.ive. 

Rel tability of Observations in Reading or Math vs. the Entire 
. Day vs the Reliability of thfe Total Study- In pa^st research using 

these instruments, adeiiuate reliability and highi>\ significant dis- 
criminations between .programs were obtained usiiig twelve five-minute 
observations for a total of 60 minutes of class time, recorded by 
each instrument across two observers and one classroom day. The 
basic data collection process prppdsed for this study would have 36 
three-minute observations for each ijistrumerit across two classroom 
days and one observer, supplemented by one additional ^ay's observation 
with two observens, for a total of 108 minutes of class time. In 
terms of past experience, this would appear to be adequate for 
reliability purposes. ^ - , 

, ^ , ' However, for the observations in reading or math, if we assume ' 
an instructional period of approximately 45 minutes, eight observations 
^ of 3 minutes each would record 48 minutes of i ns true tionit?* time acr.oss 
ty/o days and .two observers. This, then^ would be half as many obser- 
^vations and between a quarter and a^thtrd as many minutes as demon- 
strated adequate reliability for the fullday in previous work. Al- 
though these amounts seem small, we suspect that the 'number .of obser- 
vations'is more important to . reliability^ -^han the numbjer of minutes 
recorded, within limits, and that classroom behavior within the in- 
str^uction of a single subject matter is probably less variable than 
atr:x)ss the classroom day, so that aNsmaller amount of data should be 
adequate to, establish reliability although it would probably be lower 
than for the entire day/ ' • 

,But these are trade-offs inherent in the situation. The expense 
of coTlecting observational data will be a major item of the budget, 

■ . • 89 . 
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and win to a considerable degree set the sample ^ze. ,If the same 
level of reliability is sought for the observation oX reading 
instruction which would ordinarily be sought for an entire day's 
observation-, the additional days of observation required must de- 
crease the sample size. As an example, the trade-off then is between 
more reliable obseVvationaKdata for reading, or a larger total sample. 

For \he self-contained classroom, the decision which has been 
made here 'has been to seek a full day's observation by one 'observer 
on three separate days for each classroom supplemented by an additional 
observer on the- third day and accepting, lesser reliab'ility for lesser, 
segments of instruction in math and reading. While this, is not easy 
to accefit, the alternative appears to be to an unacceptable reduction 
in sample size. Several other considerations enter into this trade- 
off. Description of an individual classroom with the precision whrich> 
would be important to the evaluation of an individual teacher does 
not seem necessary; >ather, wh^t is required 1s the acctimtlTatloh of 
dat? which, ^when aggregated across classrooms, will discriminate 
between programs or settings, or identify concomitant variafnce so 
as to reduce error in prediction. Furthermore, there is some 
evidence (Brdphy and Evertson, 19741 which Indicates that reading 
g^in iss^predicted as well by observation for the total day as by' ' * 
observation or readijig instruction alone, although there were sugges- 
tions that word discrimTnatio.n gain may have been more closely as- 
sociated with observation or-reading than total day observation, 
(Whether individualization occurred in the classrooms in that study 

is unknown, but it was carried out partially in Title I schools,) 
f 
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Another part of the rationale for not seeking additional observati 
for reading and math for teachers in self-contained classrooms Is the 
expectation that the classroom environment created In reading and 
math will have'^siderable similarity to that created by the teacher 
for the remainder of the day. 

The general plan. of data collection is one In which observers 
would be trained during the fall, either In the format of a typical 
university course or in an intensive two-week workshop. The training 
would include' study of written material, lecture-discussion class - 
activities, observation of video-tape with critiqueing, and , . 
Increasingly frequent live observation in classrooms with critiqueing. 
Following training, observation of classroom environment would take 
place mid-yea^, probably between January and March, with regularly 
scheduled periods for reliability checks and retraining. 

The self-contained classroom is the simplest situation and the 
onr-which will be used as a, reference point in Identifying procedures 
to be' followed in more complex settings; but as the number of teachers 
Involved in instruction of a pupil group increases, the number of 
observations required for adequate reliability will Increase. 

Procedures for Collecting Data for Classroom Environment - A 
number of patterns for organizing instruction are anticipated and 

the data collection process and the aggregation of data for estimating 

\ 

reliabt-lity will need to differ to accommodate IfiKe different schemes. 

As stated earlier, the simplest case would be the self-contained 
classroom, either individualized or standardized, mainstreamed or 
separated » in which all pupils spend the day with the same teacher. 
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In this case, an observer would collect data throughout the entire day, 
completing a series of three observations of three minutes each on 
FLACCS, followed by a comparable series for the TPOR, and alternating 
instruments until four series of each have been collected (a total of 
8 series of 24 three-minute observation periods). Obs^ervation of 
reading and math instruction would be distinguished from observation 
during other classroom activities. After completing a series of 3 
observations on either instrument, the observer would record the amount 
of time the typical pupil spent on reading, math, or other subject 
matter, time spent waiting for the next activity, free choice activities, 
planned relaxation and lunch and recess. (NCTTE: This instrument hasn't 
been developed yet. Our previous data collection used estimates made 
at the end of the day.) On a subsequent day, a second observer would 
repeat the same observation process. 

If the self-contained classroom is individualized, a common 
organizational pattern would be a period of approximately 45 minutes 
given to reading instruction in which the teacher is available to one 
child or aJiother. In the standardized classroom^ th6 teacher might spend 
the same period of time working with three subgroups of pupils for 
approximately a third of the time each. In either case, observational 
data would be collected for the full reading periocjj, assuming that it 
is equally representative of the classroom environment experienced by 
the-typldal pupil--that being a member of a small group with the teacher 
for 15'minutes gives essentially the same access to the teacher as being 
a class meinber for 45 minutes "^(except for the differences made by 
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indtvidualization, which would be represented by the delta recording 
degree of implementat-feir of individualization). 

A more complex instructional scheme would be one in which all 
pupils leave the regular teacher for instruction in math or reading* 
In this case the observer will observe in the regular classroom until 
pupils leave for the special instruction, but will accompany them to 
the math or reading teacher or center, and will collect observational 
data there using the same procedure described earlier. As with the 
self-contained classroom, a -second observer would repeat the same 
observation procedure another day. For this pattern of organization, 
since another teacher is invol ved^whose style is likely to differ 
from the regular teacher, it woul13P be necessary, in order to obtain an 
adequately reliable measure, for each observer to observe that teacher 
teaching this subgroup of pupils another day. Hopefully, in 'most schools 
this center would serve several classrooms also in the study, so that 
the supplemental day's observation would provide data for several 
classrooms. 

The next more complex pattern would be one in which all children 
leave the regular classroom for instruction in both math and reading. 
In this case, an observer would accompany pupils to both specialized 
instructional settings, and these data would be taken as portions of 
these pupils* day, supplemented. by additional observations as was true 
of only one subject-matter outside the regular classroom. 

Still another probable pattern would be one in which a .three-teacher 
team teaches a pupil group equivalent to three classrooms. Each teacher 
1s lead teacher one week, in rotation. In that case, observational data 

•III-56 „ ^ ■ 

03 - . 



would need to be collected three different weeks, by both observers, 
in order to represent the total classroom environment experienced by 

• the typical pupil,. ' " • ; 

Another complex pattern is, one in which departmentalization" is " 
.carried beyond instruction in reading and math and pupils move as a 
classroom^ group to social studies or other instruction. In this case, 
the general pattern of environmental observation in which the observer - 

• follows the pupil group would be extended'. 

In this- case, to obtain measures of environment -for e^ach subject- 
matter as- reliable as those for the self-contained classroom would 
require as many additional observations by each observer as .there are 
instructional settings outside the regular classroom. '(NOTE: -At 
this point, it is important. to, recognize that "each such classroom costs 
two to perhaps four classrooms in total Hj ' ■ ^ 

These are some of the patterns which we anticipate meeting in the 
field and illustrate, the general princi"ple of collecting data intended ' 
to represent' the total daily experience of the typical' child. If 
other patterns are met which are not included above, the same general 
principle would be applied- in attempting to collect observational data * 
for each portion of the day which would be sufficiently reliable tp- 
permit discrimination of the classroom environment between individualized 
vs standardized and mainstream vs separate instrjicticm, and to identify 



concomitant variation! 
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III. METHODOLOGY 

B. STUDY QUESTION TWO 
^ • • 4. Data Analysts Plan 
- . a. Data Management 
The' data management procedures have been specified under 
Study Question One (see Section III A. 4. a.) ' . . ' 

b.. Data Reduction and Tran'sformatton 
FoY* Study Question Two the unit of analysts is the class. 
Therefore, all student data should be reduced to the class level by 
taking am average over students. 

The contractor will need to attend" to all the processes described 
under Section IIi:A.4.a and in addition it will be necessary to 
attend to 'the following: < ' , ' ' 

• ' Retrieve the ITTOF observational data and the.FLACc's, 
TPOR and 6R observational data, the students' pretest scores on 

the CAT tor both math an^j^Nsading and posttest scores on, HIFAM and HIFAR. 

• Scoring of HlfAM and HIFAR should be done using a program 
'such as SC0REWR2 (Rim, 1972) as the files are completed. .The 

SC0REWR2 program is capable of handling weighted scoring problems (up 
to 10 different weighting keys), and provides various test statis^tics 
including coefficient alpha. 

•/Convert the CAT pretest scores to the ADSS scale scores. 

%j Eliminate ,students .who do not have both pretes.t and post- 
,,;b«St scores on the CAT. . • 

% ' Compute residual gain scores." on the CAT for those students 
haVtng both pretest and posttest scores on the 'CAT. 
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• Reduce the individual student's residuaVgain scores 
to a class mean residual score. Individual student'a scores on^ 
^the MCI, HIFAR and HIFAM wtll also need to^be, reduced to class * 
mean scoVes by taking an average across the compensatory education 
students in the cla^s. F6r the two affective measures,^ HIFAM and 

HfFAR, an area ^rans^mation is recommended prior to taking an 
average. The observational data on the instructional treatment 
(ITTOF) will be reduced to a single scale value for each instruc- 
tional task by averaging across observations. The observational 
data on classroom climate for reading an^j^th would be aggregated 
-separately from each other and from all othersNbut, all observation 
data would also be tot.aled. Within each aggregate, item frequencies 
would be area transformed to normalize distribution and equalize^ 
variabilities. Items would then be factor analyzed and summed into 
incomplete factor scores. (Horn, 1965). A combination of MATRIX, 
TRANSFORMATIONS, and REGRESSION-CORRELATION programs in SOUPAC will. 

a 

handle these processes of data transformation, 
c. Analytic Procedures 

There are three analyses described in this section for the two. 
subquestionS of Study Question Two. They are: parti tiorying var- 
iance in multiple regression - Commonality Analysis (Mood, 1971), 
a series of stepdown regression analyses and canonical correlation. 

Prior to the separate analyses it will be necessary to test 
whether there is, a nonlinear relationship betweem independent and 
dependent variables. The analyses of variance test for nonlinearity 
(H. M. Blalock, 1960, p. 315) can be employed^^to determine if the ;^ 
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linear model holds and, addi tibnally, to .1;est the 
significance of the additional amount of variance explained 
by a non-linear modeV If there Is a non-linear relation- 
ship between dependent and independent variables appropriate 
transformations should be made prior to analysis, A square 
root, logarithmic, or some other transformation might be 
used depending on the distribution of the data. 

For ♦the analysis of Study Question Two part a 'the 
the specificity of task trejitment (ITTOF) is the independent 
variable and the class mean residual gain scores on the (1) 
CAT math and reading subtests, (2).HIFAM and HIFAR attitude 
measures, and (3) student perceived classroom climate 
measures (MCI) are the eleven (11) dependent variables. 

This st^y question asks for the unique contribution 
of each instructional task Ireatment and the amount of 
variance accounted for by some sequential combinations of 
these variables. For the former the partitioning of 
^ variance in mu'ltiple regression analyses will be used. For 
the. latter, stepdown regression analysis for each dependent 
variable is recommended for investigating the amount of 
variance accounted for by the sequential combinations of ^these 
variables. ' ' • 

Further, in order to see the relationship between all the 
dependent and independent variables, the eleven dependent 
variables would be employed as criteria in the canonical 
correlation analyses, while all of the instructional task 
treatment variables would serve as predictors. 

Study Question Two part b is essentially the same as Study Question 
Two part a except that (1) the set of classroom climate factors would be 
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added to the independent variable set and (2) the stepwise • 
regression analysis would be used. 

The stepdown regression analyses can be run on SPSS 
(Statistical Package for the Social Sciences) and the 
remainder' of the analyses can be performed by using SOUPAC 
(Statistical Oriented Users Programming and Consulting). ^ 
^ Some expected^. tables are shoy/n as follows: 

Partitioning \ilfiriance for Three Sets (W, Vj Z) of Variables 

* 







V 




(1)* Part unique to W 


.8% 






.(2) Part unique to V 




13% 




(3)' Part unique to Z 






20% 


(4) Part common to W and V ^ 


3% ■ 


3% 


« 


(5) Part common to W and Z 


8% 






(6) Part common to V. and Z 




" 7% . 


7% ; 


(7) 1 Part common^ to W, V and Z 


n 


1% 


n 


TOTALS 


20% 




36% 
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CANONICAL CORRELATION ANALYSIS 



EQUATION 
1 


EIGENVALUE 
0.7787 


, CANONICAL^ 
CORRELATION 
0.8825 


WILKS LAMDA 
0.03U6 


CHI-SQUARE 
55.4879 


DF 
36." 


2 


0.7223 




U . Ibbb 


30.6004 


25. 




0.291+6 


0.5428 


0.5637 


9.4596 


16. 


4 


0.1926- 


• 0.4389 


0.7991 


3.7013 


9. 


CRITERION HEIGHTS 
1 

VCBL -0.14126 


2 . 
-0.03130 


(CRITERIA. DOWN 
3 

0.51950 


, EQUATIONS ACROSS) 
4 

1.20597 


CHHS - 


0.72525 


-0.52465 


0,85733 


0.05642 




CMPT 


0.28166 


-0.28663 


-0.67357 


0.12355 




CPPM ' 


-0.34661 


-0.22524 


-0.24385 


-0.24820 




PREDICTOR WEIGHTS 
1 

P61 0.46729 


2 

-0.35708 


(PREDICTORS DOWN, EQUATIONS 
3 • 4 
-0.98670 -0.07439 


ACROSS 


PG2 


0.65448 


-0.06620 
-0.1^80 

• 


0.67195 


-0.80029 




PG3 

• 


-1.16949 

• 
• 


0.28097 - 

• 


0.60829 

• 
• 





P618 



0.51117 



-0.25653 




1.09339 



100 
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III.' METHODOLOGY ' 

C. • STUDY QUESTION THREE 

. 1. RatipnaTe for Selection ^of Variables > Definitions 
and Instrumentation . 

At the initial contractors conference the NIE -indicated 
that it would not be inappropriate to design plans that would 
enable one to determine how program' variables , and other variables 
of concern, relate to each other and to progranv effectiveness. 
In some ways this may be construed as an extension of Study 
Question Two. In that question, program and climate' variables 
were related to sets of cognitive and affective outcomes. In 
the present case it would be of considerable interest to 
determine how program variables together with the climate and 
affective outcome variables employed in Study Question Two 
relate to one another and to academic achievement. Elsewhere, 
we have indicated that there is a need to go beyond the direct 
comparison issue dealt with in Study Question One as the 
instructional labels "individualized" and "standardized" 
encompass variations that may themselves produce decidedly 
different outcomes. Additionally, it is inconceivable that 
instructional types alone can account for all t)ie variation 
in program effectiveness. We need to ascertain more exactly 
the nature of the conditions that do make useful particular 
factors of instructional treatment. 

The fol'lowin^^study question has been developed by us 
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as a guide for responding to this issue: 

How does specificity of the treatment . • 
of the instruotional tasks ^ along 
with other* variables of conoemy 
relate to student achievement? 

The^ variable categories selected for this study question 

are: 

1. Student Characteristics 

2. Community and School Characteristics 

3. Teacher Characteristics 

4. Teacher Attitudes and Behavior 

5. Task Treatment 

6. Opportunity 

7. Student Attitudes and Behavior 

' . _ 8. Mathematics and Reading Achievement 

The above mentioned var»iab1es were selected for a number^ 
of different reasons. For example, the variable categories: 
Teacher Characteristics, Student Characteristics, Teacher 
Attitudes and Behavior (that is, teacher performance), and 
Student Attitudes and Behavior (or student performance) 
are frequent components of teacher effectiveness studies. 
(See Koehler, 1974, for example.) In an^ earlier paper Soar 
(1974) makes the case for .^including home, comipunity, and 
school characteristics as moderating variables on the 
aforementioned components. As a result of Soar's arguments 
andVecent findings by Benrfan and McLaughlin (1975) we 
included the ca,tegory Community and School Characteristics. 

The rationale for the task treatment category is too 
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obvious for discussion, but not so with the category 
"opportunity." It should be clear that achievement gain 
must be significantly related to the degree to which the 
learner has the opportunity (i.e., time) to learn 
(Carroll; 1953 and HarniscKfeger and Wiley, 1973, for 
example). 

The variable categories listed above are operationally 



defined by the variab 



Student Characteristics 



e sets below. 



' # Percent of the "class" having had prior 

experiences with individualized instruction, 

• Ratio of boys to girls In* "class." 

• Percenl/of minority students in "class." 

• Percenpof students in "class" belonging 
to a Tow socioeconomic status group. 

• Percent of students in "class" designated 
as being educationally disadvantaged. 

• Averaged "class" intelligence quotient. 
Information for the Student Characteristics category comes 

from two sources: the Student Data Collection Form (SDCF) 
and the Lorqe-Thorndike Intelligence Test (LTIT). Both of 
these instruments are used to collect data for Study Question 
One. 

Community and School Characteristics 

• Per pupil expenditure from all sources. 

• Origin of new programs. 

• Degree of school district support for math 
and reading programs for compensatory 
education students (separately). 
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• Estimated percentage of families. assisting 
as parent ^volunteers in compensatory 
education programs. 

• Estimated percentage' of families attending 
a typical parent group meeting, e.g.,'PTA. 

• Number of years Individualized compensatory 
education math program useUJu school. 

• Number of years individualized compensatory 
education reading program used in School. 

Data from the School Principal's Questionnaire (SPQ), 
used primarily for responding to Study Question Four, provides 
data for this category. Items selected for the SPQ were based, 
in part, upon the Berman and McLaughlin (1975) studies. 

Teacher Characteristics * 

• Sex of teacher. 

• Race of teacher. 

• Number of years teaching compensatory 
education students. 

• Number of years working with the program. 



• Perceived adequacy of special program 
training. 

• Perceived degree of autonomy in setting 
curricula goals for class. 

• Attitudes toward education (progressivism 
vs traditionalism). 

Data for the teacher characteristics variable are derived 



from two sources: the Teacher Questionnaire (TQ) and Kerlinger*s 
Educational Scale-VII (ES). Both of these measures are used in 
responding to Study Question Four. The Educational Scale-VII (ES) 
is a Likert-type scale that measures two broad dimensions of 
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attitudes toward education: progressivism and traditionalism. 
The scale has been used in a number of studies and has been 
found factorially valid and reasonably reliable. One would 

jr 

assume that the most desirable teaching situation would be - 
where there is a match between a teacher's values toward 
education and instructional assignment. Any disso^nance in 
this matchup may, conceivably,' lead to implementation 
instability. 

Teacher Attitudes and Behavior 

• Teacher morale, total score, as measured on 
the Purdue Teacher Opinionaire. 

• Attitude toward program (math and/br reading). 

• Negative control vs. orderly classroom. 

• Expansive teaching. 

• Free movement and positive affect with 
little focus. 

• Teacher choice of problem. 

• Seat work without teacher. 

• Recitation. 

Teacher morale data as well as teacher attitudes toward the 

program are employed primarily in response to Study Question 
Four. The remaining data, closely related to what is meant by 
classroom climate, -are employed primarily for Study Question 
Two as well as for Study Question Four. The data related to 
teacher attitudes are collected from items .found on the Teacher 
Questionnaire (TQ) which is also used to provide information in 
respect to )^ composite variable: teacher characteristics . 
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The Purdue TeacKer Optnionaire CPTO) is designed to provide 
a measure of teacher morale. Not only does the Opinlonaire 
yield a total score Indicating the general level of a teacher's 
morale* but It also provides meaningful sub-scores which break 
down morale into some of its dimensions. The ten categories 
included are: (1) Teacher*Rapport with Principal; (2) Satis- 
faction with Teaching; '(3) Rapport Among Teachers; (4) Teacher 
Salary; (5) Teacher Load; (6) Curriculum Issues; (7) Teacher 
Status; (8) Community Support of -Education; (9) School 0 
Facilities and Services; and *(10) Conmunity Pressures. 

The Opinionaire provides specific and valid Information 
about crucial problems and tensions which concern the faculty 
and have an adverse effect on their morale. For Study 
Question Three only the total score will be employed. 

Teacher behavior data is collected by use of the Florida 
Climate and Control System (FLACCS) developed by Soar and Soar (1973), 
the Global Ratings (GR) scale, and Bob Burten Brown's Teacher 
Practices Observation Record (TPOR). All of these instruments 
have been discussed previously in the context of Study Question 
Two. 

Task Treatment is a category composed of the, ten program 
variables described earlier for Study Questions One and Two. 
Data collected on the Instructional Task Treatment Observational 
Form (ITTOF) will be accumulated over the number of visits to 
each classroom — six visits has been the suggested amount — 
in order to obtain values for each of the dimensions. In this 
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instance, the^ degree of specificity for each dimension is 
loosely analogous to the sum of the vector suras of 
instructional unit size and level of attention dimensions 
divided by the total number of visits to the classroom. 
The variables concerned with specificity of . task treatment 
are listed below. * 

4 

• Provision of curriculum opportunity. 

r 

• Statements of curriculum intentions. 

• Curriculum placement decisions. 

• Adjusting rates of Instruction. 

• Provision for individual responding. 

• Provision for Individual feedback. 

• Monitoring individual progress, 

• Performance standard for advancement* 

• Evaluation of performance.^ 

• Matching learners with next instruction. 
Opportunity , another variable category, is concerned 

primarily with teachers providing the oppc^tunity for students 
to learn. Opportunity will be affected by time as well as 
by the teacher/pupil ratio,' etc.. Some of the items in this 
category are: 

• Number of adults in the classroom, 

• Enro31ment size of the. desrignated "class" 
of learners, 

' • Enrollment size of the "regular" class, 

• Instructional grouping procedures. 
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' • Average- quantity of Instruction In 
jnathdJDatics. ^ 

• Average quantity .of instruction in 
reading. . ' 

The ipfonnation for' the above items' is derivabje from 
the Student Data Collection ?om (SDCF) lind from the Class- 
room' Description (CD) form; 'Both of thes^ Instruments are . 
employed during observational visits. Opportunity data^is 
also used for Study Question Four. / : 

Student Attitudes and Behavior data is derivable from the 
four instruments used to callect information tor Study" Question 
Two: Vty Class Inventory (MCiy> How^'l Feel About Math (HiVaM)^ 
How I Feel About Reading (HIFARy/and Florida Climate Control . 
System' (FLACCS). Items tn this catjegory are: ' . ' . : ' - ' 
• • • Satisfaction. 

• Friction. " • . ' 

• Competi ti veness . y 

< • Difficulty, ' ' 

• Cohesi veness. 

• Attitudes toward mathematics. 
, • Attitudes toward reading. 

• Negative control Vs orderly classroom. 

• Free movement an|l positive affect with 
little focus. 

Outcome variables of interes\ in Study Question Three are 
as follows: 

• Reading Achievement. 

• Mathematics Achievement, 
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In both instances we are concerned with gain scores for 
reading and mathematics achievements- The California 
Achievement Test CCAT) should be used here. 

2, Sampling and 'Selection . 

'a. Sampling Design . - 

All of the primary Sampling dimensions employed for 
Study Questions One and Two are to be used for Study 
Question Three. 

As in Study Question Two the unit of analysis is 
the class. 



3., Data Collection Plan . _ . 

Data collection pjans for most of the data that will be 
analyzed for Study Question Three hav6 already bfeen discussed in 
the appropriate sections of Study Questions One and Two. Table 
Iir.4 provides a list of nreasures suggested for use in this 
phas^ of the study. Instruments not previously mentioned include 
the following: 

Teacher Questionnaire 
"Educational Scale - VII 
Purdue Teacher Opinionaire 
School Principal's Questionnaire 
Classroom Description 

Learner measures, such as, How I Feel About Mathematics - 
(HIFAM), How I Feel About Reading (HIFAR). and Hy Class 
Inventory (MCI) are administered as outcome measures (and 
used as such for Study Question Two), but are used as inter- 
vening va^riables for this study question. 
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The Educational Scale - VII (ES) should be administered 
to teacher,s as a pretest and the Purdue Teacher Opinionnaire 
CPTD) as a posttest. As the Teacher Questionnaire (TQ) 
requests information re attitudes toward the programs, 1t should 
be -used as a posttest. Information needed earlier is 
obtainable from the Teacher Questionnaire-Screening (TQ-S) 
Instrument. 

. The School Principal's Questionnaire (SPQ) may be 
administered during the early part of the school year. • 
The Classroom Description (CD) instrument should be, 
'■used each and every t,ime an observation is made. 

4 . Data Analysis Plan . 

a. Data Management 

T-he data management procedures for this study question 
have been specified under Study Question One (see Section 
III A.4.a). 

b. Data Reduction and Transformation 

As indicated earlier, the class will be the unit of 
analysis for answering Study Question Three. Almost all of 
the data collected for this study will be used in this stage 
of analysis. Because of this, numerous and various data 
reduction and transformation techniques are expected to be 
employed. 
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The students* residual gain scores on the CAT Reading and 
CAT Mathematics subtests that were derived for Study Question 
Two will be used as dependent variables, • ^ 

Students MQ and SES, used in answering StOdy Question 
One/ will have to be reduced to class means by averaging across 
students in the designated class. The class mean scores on 
such affective measures as HI FAR, HIFAM, and MCI that were 
derived for Study .Question Two can be used directly in 
answering Study Question Three, However, it may be necessary 
^for later analyses to combine the MCI factor scores into 
composite scores by applying the first principal comporjent 
loadings as weights,-. 

Student data collected .on the SDCF (e.g, proportions 
^of boys/girls, proportion of minority group members, the 
avjsrage number of years students were in the program, and the^ 
average amount' of 'opportunity for instruction) will.be retrieved 
and reduced to the class level. Antecedent teacher variable . 
data, such as (1) number of years teaching compensatory educ- 
ation stu^lents, (2)^number of years in the program, (3) quality 
of received training, (4) sex, (5) race, and (6) philosophical 
value score on the Educational Scale measure will be reduced • 
by use of a dimensionality reduction procedure immediately after 
responses on the Educational Scale measure are scored. 
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Teacher's attitude toward the program from questions on 
the- Teacher Questionnaire and the Purdue Teacher Opinionnaire 
data will be scored by using a computer program that can handle* 
weighted scoring 'problems. • 

The contractor can use the scale scores on ITTOF and factor 
scores on the TPOR, GR/and FLACCS that were already transformed 
and reduced to. the class level for Study Question Two, Since 
the contribution of this variable to student outcomes was 
investigated in Study Question Two,-we recommend that a 
coniposfte score be derived, by applying the first principal", 
component loadings as weights. We reconnjend that a single^ 
composite score be derived for the ITTOF and two composite . 
s.cores (e.g., one for teacher affect and one for studek affect) 
be derived for the TPOR, GR,, and^ FCACCS. Data collected on the 
Classroom Description instrument and the School principal 
Questionnaire can be directly entered into the analysis, 

Through the procedures described above^-%h^_ contractor^ 
will have a score that is reduced to the c|ass -level for each 
of the variables listed in the study variabna section (III-C-1) 

,c. Analytic Procedures 
In examining the relationship of specificity of instructional 
task treatments along with other variables of concern in respect 
to student outcomes, we recommend a variety of regression, 
analyses and path analyses techniques. | 
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. The-residu^il gain scores for each of the subtests of 
^the CAT-.'Reacltng and CAT Mathematics tests wiTl be analyzed, 
separately, as 'dependent' varjables. It is especially 
important for this contractor to analyze the subtest gain 
scores, as each subtest is -designed to measure different 
• levels of the students' c6gnitive abilities. 'Pre.vit)us * » ' 
findings by Soar and Soar (1973) lead us to believe that 
the relationship of each variable of concern to one 
another and* to "the students' higher level cognitive outcomes 
might be different from thos? 'relationships formed in 
respect to the lower cognitive outcomes. * ' 

To examine the unique contribution of each independent 
variable as well as the contributions of some combinations 
of variables .of interest, we Vecommehd the method of 
partitioning variance in multiple regression analyses^. 
(Mood, 1971 )• Stepwise regressijon analysis is recommended 
for determining what special set of variables can account 
for most of the variancer 

In addition to the aforementioned concerns > the fJFP 

requests pTans ' ' ^ ^ , ^ 

"•..to include specifications for arguing causal 
relationships between program and outcome variables." 

Such a request may be construed narrowly as when "program" 

is taken^to refer only to different types of instruction; i.e. 

"individualized" aad "standardized" — or, more broadly, as 
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when the requested specifications may also include sets of 

* variables Cother than program variables) that have in the past 

been linked, directly or indirectly, to program outcomes. ^ 

We have, in Study Question One^ been responsive 

to this request of the NIE; that is, when the request is 

viewed in .the more limited sense. Because it is meaningful 

to do 'SO, we intend, in this design section, to respond to 

NIE*s request taken in jts broader context. 

*We recommend the use of path analytic techniques in 

examining or speculating about the causal relationship that 

maty exist between student outcomes and other ^variables of 

interest. Path analysis is often used as a basis for' 

inferring causality. As TatsuQka (1973) has indicated, a 

prior figurative modeling along path analytic lines is very 

helpful in sharpening the speculations and the consequent 

' avenues for testing *the resulting* hypotheses. 

The number of variables V/e recommend for inclusion in . 

the path diagram (see the hypothetical model, for example) 

« 

has been guided by the sfze of the sample studied and by » 
our estimate^ of the level of complexity that we believe can 
be reasonably managed^ This view led us to the conclusion 
that only a small number of variables should be considered. 
Such^a decision requires then' that, when appropriate,, 
composite scores be employed, 
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We recoitraend the reduction of the .njuniber or 'dqmensionality 
of the follovdng major sets of vajs.fibTfes 'b^ ^deriving a composite 
score for each subset of variaibles usij\g the ffrst principal 
'component loadings as weights 4n ordej*. to maximize the 
InterriaT consistency of the cbmpasite; measure (Lord, 1958): 
Teacher Characteristics^ (TC) ^ _ - 

1. Training and_experfence 

• Teacher -s^ercepti on of adequacy of special 
program train ijig - 

' • Number of years worki wi th the ' program 

• Npber bf^ years teacMng compensatory 
educatix)n studeots ^ ^ 

2 . kfi^tt\r^^x\tx^f varl able ' ' 

t^Tieachersj^^ position Xprogressiv ism 

' - -^7^ -v&* tradTtionalism) 

u;zz::_-t 'Teaj:ber*s perception re autonoiny in setting 
goals for class 

^_3r:£,6ex of t.eacher > 

Race of teacher 

Coimiunity and School Characteristics 

1. Principal's perception re program support 

• Principal's perception re the program 

• Principal's perception re school district's 
support - Reading 

• Principal's perception re school district's 
support Math 

2. Conwunity Support 

• Parent volunteer 

• Parent participation 

' . * III-89 : * < 
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3. Per pupil expenditure from all sources 

4. Number of years raath program used in schpbl . 
>5. Number of years reading program used In 'school 

' Student Characteristics ' , - . • 

1. Percent of "class" having had prior experiences 
with individualized instruction- 

■ 2. Ratio of boys to girls in "class" 

3. Percent of minority students in "class" 

• 4. Percent xjf students in "class" belonging to a 
low socioeconomic status group 

• 5". Percent of students in "class" designated as 
^\ being educationally disadvantaged 

6. i^vera^il^ljcla^ intelligence quotient 

Teacher Attitudes and Behavior 

1. Teacher behavior ' • 

2. Teacher morale ; 

3. Worth of math program 

4. Worth of reading program 

Task Treatment ; 

1. Instruq^tional task specificity 
Opportunity 

1. Student/adult ratio ' 

» Number of .adults in the classroom 
^ • Enrollment size of the designated "class" 
# Enrollment size of the regular "class" 

2. Instructional grouping procedures 
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3. Averaged quantity of instruction in mathematics 

4. Averaged quantity of instruction in reading 
Student Attitudes and Behavior 

1. Student behavior 

2. Student perceived classroom climate 

• Satisfaction 

• Friction 

• CompetitiveniBss 

• 'Difficulty 

• Cohesiveness 

3. Student attitudes toward mathematics 

4. Student attitudes^toward reading 

The total number of^ variables that resul\t front the above 
described a priori process is twenty-eight (28). This total 

is determined from the number of subsets (variables) within 

\ 

a variable category: 4, 5, 6, 4, 1, 4, and 4, respectively. * 
•From these a priori sets of variables, the contractor, 
depending upon the relationships determined in the common- 
ality analysis and the results of the .stepwise analysis 
contribution, may selecjt single variables or a priori 
composite variables or feven derive a new composite variable. 

The path diagram plovide here as a. hypothetical model 
does assume that we emp'^y^but one variable for each node 
in the network. 

Since we believe that Congress' Interest would primarily 

III-91 

118 



be in the 
reconinend 
model • 



relationsMp between SES and outcomes, we strongly 
that the SES variable be included in the path 



HYPOTHETICAL MODEL 




(SC) : Student Characteristics, 

X2 (CSC): Community and School Characteristics. 

X3 (TC): Teacher Characteristics- 

X^ (TAB): Teacher Attitudes and Behavior. 

Xg (TT): Task Treatment. 

Xg (OP): Opportunity. 

Xy (SAB): S^tudent Attitude and Behavior. 

Xg (OC) : Outcome. 

The student characteristics (X^), community and school, 
support (X^)' and teacher characteristics (X^) will be treated 
as exogenous variables; i.e., the sources of variation are not 
dependent on other variables in the system. The remaining five 
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variables il^ thru Xgl will be considered' either as endogenous 
or dependent variables, In the path analysis. 

The models can be modified 1f a discrepancy between 
these suggested sets and the results from using the stepwise 
regression and partitioning variance techniques should occur. 
The' path analysis technique should be employed on the 
residual gain scores for each subtest of the reading and 
mathematics test. Proposed causal relations in the model 
should be tested in order to provide stronger arguments 
for causal relationships posited for this study] (0. Amick 
and H. Walberg, 1975). 

The tables of multiple regression and stepwise regression 
analyses are similar to the tables of stepdown regression 
analyses shown for Study Qufestion Two. The product for the 
path analyses w1 lJJ>e--11milar to the diagram displayed above 
with the exception that coefficients will also be included. 

A final word- is necessary. l/^s customary to issue 
admonitions about inferring causality from either field 
experiments where full control is not possible or from 
correlational analysis intended to show both relationships. 
Path analysis -- the technique we suggest for use in answering 
Study Question Three — is^a useful procedure for inferring 
causality under appropriate conditions. However, it is not expected 
that the contractor will be in a position to^^ks^ig^HTkndomly 
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students, etc. to treatments. Under such conditions, we 
feel that the path analysis results should be viewed with 
cau ti on ."^-La brief » while. such data should be treated as 
heuristically valuable— as indicating relationships- 
worthy of further exploration— they should, more importantly, 
indicate relationships requiring confirmation under better 
controlled conditions before inferring causality. 
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III. METHODOLOGY 

D. STUDY QUESTION FOUR 

1 . Rationale for'Selecting Variables, Definitions 
and Instrumentation 



Under Study Question One we have designed procedures 
for identifying,, well -Implemented individualized instruction 
and well -implemented standardized instruction and additional 
procedures for determining the relationship between the degree 
of program implementation and student outcome variables. The 
RFP also requires that the study design provide for the des- 
cription of "circumstances under which implementation Qf 
either individualized or standardized pr6grams is difficult 
to achieve"Mf consistent implementation problems have been 
found. - This Is the intent that underlies our Study Question 
Four: 

What are the conditions that contribute 
^ ^ ' to differences in degree of ^implementation? 

^ With-the exception of two variables; 'i .e* , instructional 
type (individualized vs standardized) and instructional 
setting (mainstream. vs separate grouping) described in Study 
Question One, the rationale for^'and description of the 57 
variables used in this design section are to be found in 
Section 1 of Study Question Three. 

2. SampliTig and Selection 
The sampling design, selection procedures, and suggested 
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size for answering Study Question Four are the same as those 
for Study Question One. " 

3. > Data Collection Plan 

the Table of Instrument and Data Classification^ 
Scheme shows, no additional data collection is required in 
answering this questidn. Data collected for answering 
Study Question One part c and Study. Question Three will, be 
used herfe. 

4. Data Analysis Plan 

^ , — 

a. Data Management 



The data management procedures are the same as specified 
under Study Question One (See III-A-4-a)- 

b. Data Reduction and Transformation' > 

Study Question Four investigates the conditions under 
which adequate program implementation may be difficult to 
achieve. As degree of program implementation is to be treated 
here as the dependent variable and is to be derived separately 
for each class, the'class itself is the appropriate unit of 

analysis for this study question. The degree of program imple- 
mentation data as reduced to the class level for Stud^ssQuestion 
One may be employed here also. 

All data* §xcluding the instructional task treatment 

variables that were reduced to the class level at the first st\ge 

of -data reduction for Study Question Three," will be used in 

*• 
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answering this study que$tfon. Additionally, information 
identifying the sampling cell to which the class belongs; i^e. 
information regarding instructional typ^e (individualized vs. 
Standardized) and instructional setting (mainstream vs, 
separate grouping) will also be employed, ,(See III-A-l-a.) ' 
It should be noted that the ten factor scores from the teacher 
morale measure (Purdue Teacher Opinionnaire) will ,be used • 
here instead of the total score. Thus, the- total number of 
independent variables for this studylquestion will be 57* 

Degree of implementation information is available only 
for those classes Included in the analyses for answering 
Study Question One part c. It is appropriate, therefore, to 
retrieve data frofti classes sampled for Study Question One, 

c. Analytic Procedures 

RBS recommends, as the ana]yt1cal procedures for 
answering 'Study Question Four, the^use of multiple discrim- 
inant analyses on predetermined class tiroups. 

Sampljed classes, depending upon their degree 'of program 
implementation, should be assigned to two or three groups 
such as well -implemented and not wel 1 -implemented class groups 
or well -implemented, average implenentation, and poorly 
implemented groups. The' actual number of categories should be 
based upcM>those relations between the degree of implementation 
and student outcomes which were already 'examined in answering 
Study Question One part c. If the relation between degree of 
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Implementation and studeny^utcomes was curve! Inear, and if 
the curve showed a U-shape or a reversed U-shape» we would re- 
corrrnend the division of the sampled cl asses into three groups. 
Otherwise, two groups determined by the same cutting point . 
used in Study Question One part a and part b is recommended', 
for the discriminant analysis procedure. 

Based on the grouping, statetl above, discriminant analyses 
would be performed on the independent variables described* in 
the previous data reduction section. A study of standardized 
discriminant coefficients on a statistically significant dis- 
criminant function (two implementation class group cases) 
or on two discriminant functions (three implementation class 
group cases) will show on which variables those implementation 
groups ate different from, each other. _ - - . ^. - 
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IV. LOGISTICAL PLANS ' ' • 

A. ; PROJECT ORGANIZATIONAL STRUCTURE • 

Thr«fi major task areas of the District , il Survey have 
•been Identified as" the basis of the proposed organizational 
.structure. These as shown in Figure IV. 1 include (1) Training, 
(2) Field" Work, and (3) Data Processirtg and Analysis and as 
such comprise the three major work units of the proposed 
project organization. In addition to the major work Units, 
the proposed organization includes a Recruitment and 
Staffing proup, a Research and Implementation Group, and a ■ 
Monitoring and Quality Control Group. 

The responsibility for the overall project management. 
1s the Project* 61 rector's as shown in. Figure IV.l. 
This person would-be responsible for all major decisions, " 
allocation of staff resources, assignment of accountabilities, 
financial control, and setting of personnel policy*- The 
Project Director would also be the principal inVi^\tigator 
forJ:he study taking a major ro1e_ in the analysis and 
preparation of the f'inal report. 

The organization. and staffing required for the study is 
directly related to the.stud^- time schedule shown in Figure IV. 2 
It has been assumed that the NIE time line will permit the 
study to begin on February 1, 1976 allowing a full seven 
months for the initial preparation phase prior to the 1976-77 
school year. The actual in-school study phase of the study 
will run from September 1976 through April 1977. The Analysis 
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and Reporting Phase will overlap the "in-school" study 
phase starting with the processing of prabest data during 
October 1976. / 

The duties of the primary or core sta?rwJl require., 
specialized professionals with considerable training and 
experience. It will be necessary- for these individuals 
to complete their initial assignment during the Preparation 
Ph&se in order to move into other major monitoring and 
writing assignments during the field Work .and Analysis 
phases. The preparation phase will include a recruitment 
and training effort for both full- and part-time 'staff 
in order to provide classroom observation teams, test 
administrators, and data collectors. ^The following sections 
will provide some detail regarding the proposed organizational 
work units, staffing requirements, nature and schedule of 
assigned tasks, as well as level of expertise and experience 
required. 
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1. Recruitment and Staffing Group 

The major work of this group is scheduled, between the 
beginning of the study and wilTlast through July of 1976 
at which time staffing for primary staff, classroom observers, 
test administrators, consultants and the -National Advisory 
Review Committee will be completed. (See Figure IV. 3.) In 
addition there .may be a need to recruit additional test 
administrators in February of 1977 to replace staff who drop 
out of the study after the fall testing. (It has been 
anticipated that approximately 25% of the test administrators 
who participate in the fall testing will drop from the study • 
because of changes in employment or availability.) 

A primary concern of the entire recruitment effort^-wi-ll- - 
be to obtain as many qualified minority staff members as V 
possible. To accomplish this goal, recruiting activities ^ 
will be focused through minority employment services', :^ 
university placement services where a signi^ficant number of ^ 
minority groups may be enrolled, as well as various minority 
publications. Such publications might include The Black 
Scholar , The Journal of Negro Education . Integrated Education , 
and the Newsletter of the Black Child Development Institute . 
Contacts might also be made through minority caucuses of 
various national education groups and local offices of the 
Urban jLeague in areas where study centers are being considered. 
This policy is warranted in as much as the study will be 
concentrating on populations where there is a high incidence of 
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minority groups* 

A major criterion for employment in this study will be 

successful experience in working with educational programs 
for children from poverty backgrounds. This criterion will 
be used in screening applicants along with other educational 
and experience criteria shown in Figure IV,4. 

The Recruitment and Staffing Group will also handle 
matters of a personnel nature for the project including 
office space arrangements for project personnel, relocation — ^ 
of new staffTand notification of unsuccessful applicants. 

This first task of the -Recruitment and. Staffing Group 
will be to complete the primary staff VThis will include 
identifying .and hiring, where necessary, four Assistant 
Project Directors, the Research and Implementation Staff, 
the Training Staff, as well as support personnel. It is ^ 
assumed that most recruitment for these jobs will have to 
be initiated by the contractor prior to February 1976 arid 
will need only a short time to complete. The first ^task of 
major proportion is to locate and contract about 30 con- 
sultants to conduct the screening of potential study sites. 
Since this will be done during March and April requiring 
orientation and training during- the last week of February, 
the time line on this ta^k will require a rather intensive 
effort. However, since it is anticipated that much of the site 
screening can be done very effectively by college and university 
professors^ and their graduate students, the contractor should 
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Figure IV, 4.- Screenfng criteria for study staff 



Primary or Core Staff 

(1). Successful experience In working with educational 
prpgrams for children from poverty backgrounds 

■(2) Successful experience with individualized 
educational^ programs' 

(3) Doctoral level education or equivalent experience 

(4) Experience with observational and survey research 
tool s 

(5) Experience with formal educational evaluations 



Classroom Observation Staff 

(1) Successful experience In working with educational 
programs for children from poverty backgrounds 

(2) Enrollment In a graduate degree program: Masters 
or beginning Doctoral level 

(3) Succ3s«f«4 classroom teaching experience or use 
of observational or survey research tools 



Test Administrators 



(1) Successful experience with 'educational programs 
for children from poverty backgrounds 

(2) At least one year of successful teaching or 
substitute teaching experience 

(3) Enrollment In a graduate program or employed as a 
. substitute teacher In a participating district 
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anticipate focusing recruiting contacts oH universities and 
collegfes of education in .metropolitan areas near anticipated 
study centers V 

The second maior task will be to staff the field 

operations group with 17 classroom observers e^nd^ four field 

/ 

supervisors. .This task's major difficulty. will be attracting " 
high quaTity people to a job lasting less than one' year. 
Because these positions will involve considerable travel and 
will be of comparatively short duration, it is recommended 
that an attF^ctive salary and benefits package be considered'. 
It is also recommended that recruiting be centered' in areas 
where study sites a,re apt to be located thus avoiding re- 
location costs as well as extensive costs .for lodging and - 
travel to and from home, during the study. A final Incentive 
to be offered to graduate student applicants would Jae an 
arrangement to g"ive graduate credit Tor the training and field 
work experience. 

The final major staffing task is the employinent of test 
administrators. Since this position is part-time, calling for • 
as many as 11 days in the fall and TO 1/2 days in the spring, 
the recommended primary target for recruitment will be sub- 
stitute teacher staffs from school districts participating 
in the study. A secondary target would be advanced students 
-enrolled in graduate schools of education. 

A minor but very important task early in 'the study will 
be to establish a National Advisory Committee for the purpose of 
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reviewing the study plans, -instruments, and reports. The 
group win provide essential external review at several 
stages. Recruitment will be based on nominations made by 
the NIE, USOE Title I office, Right to Read Program office, 

. and other similar offices. . Conmittee members will be 
selected^on the basis of their expertise in either compen- 
satory education, individualization, or educational research ' 
methodology with two being chosen froiji-each area. Final 

.screening will be done by the principal study st^ff based 
primarily on interest, availability and willingness to serve. 

It is anticipated that 20 full- or part-time staff -personnel 
17 classroom observers, ancr97 test administrators will have 

IK- 

to be recruited. Because of the continuing poor job market 
situation in education it is expected that approximately - 
10 to 15 applications will be received for each position. 
It is anticipated that recru1,tnient and staffing processes 
will require one professional working full time for six 
months as well as ten other professionals working 10% time 
in screening and 'interviewing procedures over the five month 
period or a total of 3 person-months. The full-time pro- 
fessional will report directly to the Project Director. A 
full time secretary will also be needed for this effort for 
the six months. Following the completion of the staffing,, 
both full time staff personnel will be reassigned to duties in 
the .data processing and analysis group. 

Figure IV. 5 contains a-dmplete listing of the various 
■ subtasks for the entire staffing and recruitment effort. 
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^ Figure IV. 5. Staffing^ and Recruitment^ Ta'sks 



1.0 Complete Primary. Staff 

1.1 Staff Site Screening Consultant Group 

1.2 Establish National Advisory Group \ ' 

1.3 Prepare Job Descriptions and Selection Criteria for 
Classroom Observers, Jest Administrators, etc. 

^ 1.4 Preparation of Recruitment Brochures, Announcements 
and Advertisements \ 

1.5 Recruitmen^of Classroom Observation Personniel^ 

1.5.1 Send Recruiting Materials ^and Announ(?ements 
to Placement Agencies 

1.5.2 Develop File of Applicants 

1.5.3 Conduct Screening, Interviews, and Selection 
Process 

1.5.4 Complete Hiring Procedures for Selected ^taff 

1.6 Recruitment of Test Administration Personnel 

^1.6.1 Solicit Nominations from Substitute Teacher 
Lists of Participating School Districts 

1.6.2 Contact Nominated Applicants by Letter 
Containing Application Forms 

1.6.3^ Send Job Descri])tion to Graduate Schools 

Located in Area of Selected Sites if Substitute 
Teacher Approach Does Not Yield Enough Qualified 
Applicants 

1.6.4 Co)kluct Local Interviews for Interested Applicants 

1.6.5 Complete Review Process and Make Selections 

1.6.6 Secure Personnel Contracts for Selected. Test 
Administrators 

1.7 Establish Personnel File of All Project Staff 

1.8 Attend to Relocation Concerns for New Staff 
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2. Research and Implemeptation Group 




This group will operate during the entire study with 
major responsibilities occurring during the Spring of 1976 
and the Spring of 1977. A primary concern of this group 
will be to assist the Project Director in the proper imple- 
mentation of the study design. This group of highly capable 
educational researchers will assist the Director in pro- 
viding leadership. The primary responsibilities of 
group will be the pilot test, the final analysis of- data 
and the preparation of the final report. -This wiii include 
the field testing and modification of the classroom 
observation instrument for measuring program variables, the 
ITTOF, The group will test out the proposed study design 
and implementation procedures during the pilot test in the 
spring of 1976. The group will recommend and make any 
necessary modifications in the procedures, coniruct~-a major 
/-"^re of the final analysis and prepare major sections of 
^ the final report. The group will work closely with the 
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other work unit leaders and the Monitoring and. Quality 
Control staff. The scheduled major tasks of this group are 
shown on the" time line in Figure IV. 6. A listing of subtasks 
is contained in Figure IV. 7. 

It is anticipated that the primary staff requjTped. for 
this group will be as shown on the next page. 
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• 1 expert In individualized instruction 
753; time for 18 months 

• 1 expert in research methodology 
7S% time for 18 months 

'• 1 ejcpert in compensatory education 
7S% time for 18 months 

In addition to these primary staffs it will be necessary 

to have staff from the Training Group work half time with 

the field testing of {he ITTOF and the pilot test during 

* • 

the months of February, March, and April. This work, however, 
will dovetail fnto their training assignments and the split ^* 
assignment should hot be a problem. 

. While the primary members of the Research and Imple- 
mentation Group will work only 75^ time, they will. work full 
time during the pilot test in the spring of 1976 and again 
during the analysis period in the spring of 1977. They will 
have reduced Research and Implementation duties during the 
fall of 1976 and winter of 1977 moving into other supervisory 
and monitoring positions. During the pilot test they will 
work closely wtth the Project Director and will have the 
assistance of principal members of the Field Operations Group 
and the Data Processing and Analysis Groups. Each member of 
the Research and Implementation Group will report directly 
to the Project Director. 
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Figure IV. 7. Research and Implementation Group Subtasks 
Page 1 of 2 



2.0 Testing and ModificatiorTlrfv ITTOF * ' 
2.0.1 Arrange for Test Classrooms 

• ' 2.0.2 Train Staff to Use ITTOF 

2.0.3 Conduct Observations 

2.0.4 Evaluate Observation Reliability and 
Useability 

2.0.5 Revise Instrument 

2.0.6 Recycle if Necessary 

2.1 Conduct Pilot Test ^ 

2.1.1 Select Sites for Pilot Test 

' 2.1.2 Obtain Necessary Jo^t Agreements and 
Community Support for PiOot Test 

2.1.3 Conduct Tryout of ITTOF Obs;ervation 
Procedure 

' 2.1.4 Conduct Tryout of Classroom Climate 
- . Observation Procedures 

2.. 1.5 Conduct Tryout of All Interview Schedules 
to be Used in S'tudy 

2.1.6 Conduct Tryout of Other Data Collection 
■ Procedures 

2.1.7 -Conduct Tryout of All Posttesting 
Procedures to Classroom Groups Observed 

■ in Pilot Test 

2.1.8 Conduct Tryout of All Data Processing and 
Computer Analysis Procedures 

' 2. 3-. 9 Review All Study Procedures Based Upon 

Results of Tryouts and Make Recoimiendations 
• for Revision 



IV-17 

143 



ERIC 



Figure. IV. 7^ 
Page'2 of 2 



2.2 Review Recommendations for Revision of Study Procedures 
With National Advisory Committee and the NIE 

2.3 Make Necessary Revisions to Study Procedu/es Resulting 



From Pilot Test Following Approval of Nat 
Advisory Coirmittee and the NIE 



onal 



2.4 Review1>roblem Reports and Recommendations for Pro- 
IrZp Q"ali<^y Control and Monitoring 

2.5 Along with Project Director Review Computer Analysis 
Of Study Data and Reconmend Further Analyses Whenever 
Warranted 

2.6 Along with Project Director Prepare First Draft of 
Final Report • 

2.7 Revise Draft of Final Report Based Upon "Review by" 
National ^Advisory Committee and the NIE 
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3. Training Group 

The training group will operate during the eleven 
months starting from February of 1976 with the major ac- 
tivities being cewpleted during the first eight months. 
(See schedule of major training tasks in Figure IV. 8.) 

The primary concerns of the Training Group will be 
developing and conducting four training programs. The 
training programs will be d*^veloped for (1) the screening 
site consultants, (2) thQ, program implementation^observ- 
atioa teams, (3) the classroom- climate observation teams, 
and (4) the test administrators. The development- of each, 
of the training programs will follow a traditional devel- 
opment sequence shown in the listing of Training Subtasks ' 
in Figure 

The training for program implementation ol?servations 
will occur during vjuly and August of 1976 culminating with 
trial observations during late September and e^trly October. 
The trailing will ^include apprpximately 15 hours of 5lmu- ' 
lated classroom observation tioie and 10 hours of actual clas 
room observation time. The observers will ^Iso receive 
training in test adninistration and will, in turn, be 
expected to train the local test administrators. It is 
estimated that the entire training program for classroom 
observers will involve 50 to 60 hours of classroom instruc- 
tion in addition to fairly intensive observational work in 
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Figure IV. 9. Training Group Tasks 
Page 1 of 2 



3.0 Prepare Training/Origntation of Site Screening , 
Consultants 

3..1 Conduct Training/Oriehtatioh of Site Screening 
Consultants /^^x. \ 

3.1.1 Arrange for Needed Training Facilities 

3.1.2 Conduct Training 

3.1.3 Evaluate Effectiveness of Training 

3.1.4 Review Evaluation Results and Recoinmend • 
Termination of Trainees Not Meeting 
Acceptable Criteria 

3.2 Develop Training Program for Program Implementation 
' Observations 

3.2.1 Conduct Needs Analysis ^ . 

3.2.2 Prepare Objectives 

3.2.3 Prepare Criterion Measures 

3.2.4 Develop Lesson Plans and Learning Activities 

3.2.5 Develop l^arning\Materials 

3.2.6 Obtain Logical Critical Review of Training 
Program Components anov Revise Accordingly 

3.3 Develop Training Program for glassroom Clima.te 
* Observations 

3.3.1 Conduct Needs Analysis 

3.3.2 Prepare Objectives 

r 

3.3.3 Prepare Criterion Measures 

3.3.4 Develop Lesson Plans and Learning Activities 

3.3.5 Develop Learning Materials 

3.3.5 Obtain Logical Critical Review of Training 
, ' Program Components and Revise Accordingly 
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Figure IV. 9 
Page 2 of 2 
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3.4 Develop Training Program for Test Administrators 

3.4.1 Conduct Needs Anal^^sis ' 

3.4.2 Prfepare Objectives 

3,4^,3 Prepare Criterion Measures 

3,4.4 Develop Lesson Plans and learning Activities 

3.4,5.. Develop Learning Materials 

3,4,6 Obtain Logical Critical Review of Training 
-Program Components and Revise AccorUingly 

3.5 Conduct Training for Program Implementation 
* Observations 

3.5.1 Arrange for Needed Training Facili-ties 

3.5.2 Conduct Training ^ ' - 

3.5.3 Evaluate Effectiveness of Training' 

3.5.4 Review Evaluation Results and Recommend 
Termination of Trainees Nat Meetijig- 
Acceptable Criteria - 

3.6 Conduct Training of Test Admi^/iistrators • 
3,6,1 Arrange for Needed Tmning Facilities 
3,6:2 Conduct Training . • 

3.6.3 Evaluate Effectiveness of Training^ 

3.6.4 Review Evaluation Results and Recommend 
' Terrtii nation of Trainees Not Meeting 

AcceatSrt)le Criteria 

3.7 Conduct Training for Classroom Climate Observations 

3.7.1 Arrange for Needed Training Facilities 

3.7.2 Conduct Training 

3.7.3 Evaluate Effectiveness of Training 

3.7.4 Review Evaluation Results and Recommend 
Termination of Trainees Not Meeting 
Acceptable Criteria 
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sunwer school classes, examination of program materials, 
and viewing of films and video tapes. It is recommended 
that three classroom observer alternates be included in 
the training in the event that" substitQtions neeji to be 
made among the 17 selected candidates. While it is not 
necessary, it is recommended that the training be conducted, 
as part of a program of graduate study for which observers^ 
receive graduate credit. 

^ Since the observations of classroom climate will be 
conducted only during February and. March of 1977, the 
training for the classroomicl iitiate observation teams will 
occur during late November and December of T976 again culmin- 
ating with a series of tH^V 'Classroom otbservat ions The 
training time required is approximatelyMO hours and 20 hours 

of classr^Qom abser\/ation- Again while it i^s not' necessary, 

' ^ . ' . ^ , , 

it 1s rfecommended that training for cTassr^om^ climate observ- 

ation be part of a 'program of graduate 'Study. 

The training of test administrators is exp'ec-ted to be 
a maximum of 4 hours of instruction. Since the test admin- 
istrators'will be from the locale of the test site, training 
will occur 'in centralized sites within 25 miles of ^ the test 
site whenever possible. Training classes for test administrators 
will be small (5-10) and will be conducted by the classroom 

implementation observers who will assist in monitoring and 

* , 

administration of pre- and posttests, . . ' 
' . IV-23 
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The training group will also be responsible for the 
orientation and training of the screening site consultants. 
As mentioned- earlier, this group will work on a part time 
basis visiting prospective study sites located in the 
geographical centers being considered for the study. Since 
the screening teams will be recruited from university and 
college of education staffs,, &nd will, as prerequisites 
for their a^ssignment, be familiar with the notions of ' 
individualization, compensatory education, and classroom 
climate, it is anticipated that orientation and training 
will be accomplished in three full day sessions which will ^ 
introduce them* to the study and the specific precedures 
and instruments- tg be used in the screening process. Training 
will include several sessions of role playing^using video 
recording and playback equipment. Each consultant team will 
have at least one opportunity to be taped and critique his 
or. her own tape during the training sessions. 

The training organization and staffing requirements are 
shown in, Figure IV. 10. The Assistant Project Director for i ' 
Training will report to the Project DireAor, The special 
prerequisites for the developer/trainers would be successful 
experience in providing in-service te>cher training or college 
teaching in the field of education. Following the completion 
of all training tasks, the training staff will be reassigned 
to the Monitoring and Quality Control group in order: *to provide 
continual support to observation teams as they perform their field 
work. This will be discussed further in the Quality Control section 
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4.. Field Operatiojfis Group ' ' , . 

The workscope of the Field Operations grouj) constitutes 

the largest set ^of activities of the study and consequently, 

involves the^most staff. Figure IVJl shows' a, time table of 

the major activities. for th0 .Field Operations grpupVith 

two^major subrdivisions: .{4.0> Screening and Contracting, 
■* * * « ' 

'and'^(4'.T) Classroom Observation and Data, Col lection. Because 

of ihese functional groupings', tHe Field, Operations group ^ 

will be organized into two*operati^g groups as shown in 

Figure iV. 12. ^ • , ^ 

• " a. Screening an3v Contracting Group 

Th6* Screening and cbntractin^g.phase, a mo^t ^critical 

period 'of fiye months at the begi,nn.ing of the -study, will 

involve extensive telej)h6ne, Contacts aftd^rvisits to -prospective 

study sites. The primary concej^n during the .scraaning^ and 

con^l^ractfng phase is ^ secure the cooperation of sufficient 

.sites in which to conduct the study, «^ While screening is a 

major part. of the work, ,the contractor. my$t. also emphasize 

the^positive-ben§fi"(:S'.of the study to the prospective partic- 

ijpantsj be they teacher '^rpups, parent groups , 'community 

groups, school principals^, school boards! central office 

'superintendents... \r\ this regardj-one of the^arly tasks of 

' ' ' t., , . ' 

the ^Sp/eening group. will 'be the prepar.atioh of v&^ious public 

information brachui^es* designed for several different target 

growpjs with, varying leveU^of sp'^ctftcity. IV. 13'. 

sumiJtar.izes th^i brochures, that vyf 11 be neededL 



to 

•I/) 
OS 



o. 
o 

to 
c 
o 



<4- 
O 

<V 
O) 

x: 
u 



J 



V2! 



r- U. 



C 

o 



erJc 



$2 



O 

o 

z 
o 

Ul 

a. 
o 

o 































5 
























«) 












on 






























€1 


























3 








(A 


i. 
























•o 




(A 






€1 








M 
















«> 




4-» 




o 






o. 




€1 








c 




> 




JC 




C 


o. 




4J 


t 


=> 




♦» 








o 








u 




C^ 




c 


O 




1 


















c 


CO 




E 


\ 


o 




lA 


















u 


o 




(A 


4) 


3: 




4-» 




o 












O 




< 


*r- 




E 


o> 


o 




U 


o 








3 


' C 




€1 






4^ 


u 


40 


c 




40 


4) 








€1 








r-* 






U 


o 


€1 


40 










o 






U 


c 




01 




1. 


€1 


:t 




&. 


o 


c 


r~ 




u. 




Id 


o 


01 




«✓) 


(A 


o 








1. 


u. 


4) 


O 


«J 




c 


c 


1. 


41 








o. 


r~ 


•a 


c 


< 






O 


E 


•a 






CO 


1. 


c 


E 


C 


Q. 


O 




o 




•0 


1 








4> 


E 




L> 




o 


01 


3 


O 


€1 


•r* 


cn 






•a 






U 


o 


€1 




c 


o 




I/O 




•r* 




c 


cn 


ft. 




o 


c 


Id 


z 


> 




0) 


i. 


<u 




40 




Id 




c 


E 














4> 




(A 


01 








i. 


> 




»— < 








4-» 


H- 




C 


j: 


(A 


i. 


^> 


Id 


•o 


01 








I 




U> 


c 


O 


Q. 


o 


u 


<a 






o 


01 


o. 


-J 


(A 


E 




o 


01 


o 








I/O 




< 


c 






o 






40 


> 


u 




o 




1. 


o. 




O 




3 










H- 


1. 


1. 


if% 


1 




O 


U 


€1 


0) 








-c 


«o 


"O 




1 


cn 


01 


if% 


<M 


"O 


O 


(A 






•0 


C 




40 




r~ 




01 


o 


-M 




(A 


c 


a. 


01 


01 








o 




OJ 


€1 


0) 


1. 




C 




O 


4d 




o 


H- 


I/O 


01 


o 


o 


c 


O 




> 


a. 


a. 




o 


Ok. 












•M 






o 






10 












o» 




€1 


•M 








4-» 




41 
















c 




1. 


U 


u 


«✓) 


01 


u 




1. 


c 




u 


u 


u 


u 


O 




E 


10 


3 


3 




i. 




40 


40 






3 


3" 


3 


3 


3 


c 


«> 


Q. 


•o 


-D 


01 


3 


% 


> 


o. 




€1 




-D 




•o 






(A 


01 




C 




u 


c 


1. 


4> 


(A 




C 


C 


C 


c 


C 


CI 


(A 


1. 


o 


o 


40 


01 


o 


41 


1. 


(A 


10 


o 


o 


o 


o 


o 


1. 


< 


a. 


<-> 


o 


ac 


i/> 


o 


(A 




< 


ac 


o 


o 


o 




u 


u 


































v> 
















o 






















CM 


CO 




in 










CM 












00 


o 


d 


d 


d 


d 


d 


d 


d 




• 














































• 







































lV-27 

153 



Figure IV. 12. Field Operations Organization 



Assistant Project Director 
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gure IV. 13. Information Brochures : Designated by Target Grdup 



(1) state Title I Director, Distri-ct Superintendent > 
District Central office, Principal 

•(2) Teacher, Teacher Organization 

(3) . Community Group, School Boards, General Public 

Information 

^- 

(4) School District Research Review Committee (an 
abstract of the proposal is anticipated here) 
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Ttie first ma jbr screening task will involve identification 

and plotting 9f potential study sites on a map. .Areas of high 

concentration will be identified as potential stildy centers. 

Because of the logistics and cost problems that would be- 

encountered in a nationwide sampling of schools and also since 

the NIE has not required a geographical sampling » the screening 

process will concentrate on identifying sites within -a 

close proximity of each other. Preliminary work with the 

screening process during the planning phase ijadicates that 

severeO clusters may be possible.' One tlUster is the 

northeastern section of the United States centered 1n the 

New York^ Philadelphia, Washington corridor. 7\ lecond 

* I. 
possible cluster might be centered in the CMcago area/ 

with a third in the Pittsbi^gh area. 

Once the potential study centers have been ident^fied» 

state ESEA Title I Coordinators will- be contacted for the 

states* In which- the prospiective study centers are located 

in order to- determine per pupiV expendi ture for compensatory 

education in each prpspectiv^e school district. This contact ; 

will aTso serve'as a proper protocol • contact^ to inform the 

state ^coorclinatorv of the study, the intentions of the conb^to 

and NIE. Eaph of these contacts will ,be\ made initially by * 

phone or personal visit. . Descriptive brochuj^es- will be y 

provided- in letters of thanks following the contact. 

' Fqllowiag this initial contact the^screening of individual 

school districts and sites wiTl begin. ^^ ... 

' ' ' • ' rv-30 



Ustng a telephone screening process, school districts and 
schools will be contatted and screened with respfect to the 
remaining screening variables. 

.Telephone calls will be made initially to the school , 
district central office followed by calls to the individual 
school principals. In the case where a specific individualized 
school site has been identified, the call to the district ' 
office win be to secure nominations of comparable standard- 
ized sites as well as 'to serve as a protocol step to con- 
tacting the identified site. In the case where only- a 
district has been identified as using individualized progranvs 
the call to the district office will be to. seek nominations 
and permission to contact individual school principals. 

During the contact to the district office, itsho^ld^be 
as^certained if the dis^tr^ict Vesear^h-policy_ entails approval 
of all outside research' by a research review committee. 
The experience with the screening procedure during the planning 
phase indicates that^many of the larger school' districts in 
metropolitalrareas have a research review committee which must 
approve all research studies. In most cases the time to 
obtain approval is at least 30 days. In order to avoid delaying 
the screening process, the contractor should make the initial 
screening contacts, to large metropolitan districts at the 
earliest possible time and submit appropriate requests for 
approval of the siudy. This request or proposal will need to 
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contain detailed specifications of ^11 procedures' and iftStrumehts 
to be used. 

The telephone contacts to the district' and tKfe school 
principal vlfll be done using telephone interview schedules ; 
(See Appendix) in which specific. questions pertinent to the ^ 
screening criteria will be asked. Data will be recorded by 
^ the interviewer on the interview response sheet fpr later . ,r 

review. 

Because of the heavy reliance on telephone interviews 
" . for the screening process the contractor should have secured 
sufficient Wide Area Telephone Service -fWAXS) arrangements 
since it i? estimated that as many as 1000 sites would be 
contacted by long distance telephone during the months of 
February and early March. (See facilities section on 
recommended telephone system.) The tryouts of the interview ^ 

-schedul e i ndi cate on- the- average-that the-J^lma needed is 

approximately one half hour per screening interview. This 
will require approximately 500 hours of telephoning by five 
professional staff persons functioning as telephone interviewers 
fpr five^ hours per day five days a week for four weeks. 

Immediately following each initial telephone screening 
interview a letter of thanks and a descriptive brochure 
explaining the study and Its intended benefits will be sent 
to the school principal and/or the district superintendent. 
After reviewing the results of each teTephonis interview, 
sites meeting the selection criteria (either individualized * 
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or standardized) will be sent a second letter Indicating that 
the site is being seriously considered for inclusion in the 
-study. Also included in the letter will be information des- 
cribing the study, what specifically it. would niean to the 
local district in terms of obligations, benefits, and incentives. 

The , results of the phone screening will be revieV^ed and . . 
initial selections wilT be made for on-site screening. At 
this point all schools will b6 notified of their 'status .in- 
cluding those selected for on-site screening and^tbose which 
,have been eliminated. Arrangements will be made for site 
visits by phone followed by a confirmation letter which will 
-include a packet of materials describing the study,- its specific 
benefits to the school, the teachers and the coiTinun.i ty. " ^ 
Also included will be an outli'ne of those things to be 
completed during the site visit; i.e., principal interviews, 
classroom visitations, teacher interviews,- and a meeting with 
appropriate central office personnel. Outlines of the major 
points to be discussed in these meetings will be included with 
a copy of the prirtcipal and teacher interview forms. 

Screening visits sliould b'6 considered as having several 
major purposes: (1) gaining initial interest and support of 
the school district personnel for participation in the study, 
(2) gathering needed information for the screening process, 
(is) obtaining information that would be helpful in assessing"- 
the difficulty of obtaining community support should the site 
be selected, and (4) cohtac ting -appropriate district contracting 
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officials to "determine the proper procedures for. processing ^ 
the joint agreements in the event the site is selected for 
the ^tudy. 

Screening visitations will be scheduled" at schools in 
close "proximity and should take at least a, full day per 
school. Visits will'^e made by teams of two. study personnel, 
one to meet with the principal and administrative staff an'd 
the second' to visit the prospective classes and to inter- 
view teachers. Where iDultip.le potential sites exist in a 
single district, there may be some savings in time with 
respect to meetings with central office personnel. However, 
it'is-antic-ipated that this savings in time will be offset 
in large- urban districts by the increa.sed number of central 
'office personnel to be contacted "and the number of protocol 
visits that may^be necessary. '. " 

Following the site visits, potential sites will be 
reviewed using' the selection criteria and sampling procedures ^ 
discussed-earlier. Following the selections of desired 
study sites and ultimate sites, all schools visited will be 
informed of their sta.tus by letter. Phpnfe contacts will' be 
.made to the first choice study sites and joint agreements will 
be prepared and sent to fhe appropriate school pfficials 
determined in prior visits to' the district central office. 
The joint agreements will outline the specific obligations 'of 
. the contractor,, the school district, "^the principals,' and the 
teachers. It will also contain specific data regarding incentive 
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pa)«nents for all time spent by district personnel in regard 
to the study. The mailing '"of joint, agreements will be 
followed up by calls- to facilitate -the acceptance of the 
district approval process.. Substitutions will be made from 
the "alternate Mist wheaever a school decides to decline the 
s^nvitation to participate. " ' 

the' first indibation of approval by a district, 
plans fhr the development of comituni ty support will be 
-initiated for the district. This will consist of calls to- 
school -community leaders, PTA'^s and other influential 
community groups. Presentations .wil 1 be made to executive 
or full group .meetings when it seems advisable. This process 
will continue from April to June and again in September 
through November of 1976. 

It is estimated that as many as 300 screening site 
visits will need to be made during March and April of 1976. 
Since the visitations will coincide with spring recesses 
in many schools', this period will proyide only 40 good 
visitation days (avoiding two days prior to and two days' 
foflowingvyacations) which means t}iat on an average about 10 
sites will need to be reviewed per day. This will necessitate 
15 teams, of 2 site visitors making an average of 2.5 visits 
each week.. -Several people jcoordinating vistts from the 
"contractor's home office will also be necessary. Site 
screening teams will need to have sufficient background exper- 
ience to conduct the screening and be successful in meeting 

ff 
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with teachers, administrators, and superintendents. Since 
they must be able to explain in detail the nature of the 
study with little training they will need to be a group of 
people with high quality backgrounds in educational research 
with experience in' supervision of instruction as well as in 
dealing with teachers and administrators. 

The Screening an^ Contracting group will also need to, 
have several people to handle. closing the contractual 
arrangements with the school districts. This staff will ^ 
more than likely be the same staff who arr'aaged for 
visits of the Screening staff. Following the screening 
visits, several people will need to' engage in developing 
community iWolvement. These people- will more than likely 
become the classroom observation supervisors during the 
1976-77 school year. 

b. Classroom-Obseryation, Testing. and Other 
, Data Collection Group 

The major responsibility of this group will be to 
schedule and conduct all data collection activities including 
interviews, testing of students/ conducting classroom observ- 
ations,, as well as to collect other pertinent data from school 
and district records. The first major task of the Classroom 
Observation and Data Collection group will be the^formulation 
of a detailed schedule of testing , observation, interviews, 
and other data collection."^ This task is to be attended to - 
only after^ joint agreements have been secured with all the 
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dtstrtcts. selected In .the study sample.. Sctieduling will be \> ; 
done in cbnj;iinctton with the assignment of classroom observ- 
ation^ staffs .and will take into' account available staff : 
resources in an effort to maximize i:he coverage at a minimum 
cost for travel and accoimodatiom. 

i. Test Administrations- Pretests for all 
students participating in the study wiTl b'e administered 
during the period of September 20 to October 1^ 1976 with 
posttests admiriistered during the period of April 18 through 
April 1977. ' 

Test/admini strati on in the study classes will be conducted 
by a test administrator Working with the assistance of the 
classroom teacher/- The total testing time will' be- about five 
hours during the week for each child. Schools using. IGE-type 
(large group) classes wiTl require an additional test administrator. 
Student test sessions will be, kept as^ short as pos'sible with 
the test administrator^ moving from class ^to class within schools 
where more than one class* is participating in the study. 
Figure IVJ4 shows the testing patterns that will be used subject 
tV minor adjustment? due to school requirements. <> 

The test administrator will Ireturn to the school' during 
the week following the main testing to test students who were 
absent from class during the^ testing. 

Prior to pretestings test administrators will fill in 
student name and olher identification data on answer .sheets 
in order to save time. This will help in identifying absentees 
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Figure IV. 14. Study Testing Patterns 
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'and tn thfr processing pf data: Student names will bfe' taken 
""from the teacher's class list; ! ^ • * * : 

' ' ^ ^' : %^ : ^ ' ^ ^ . . 

,For posttestihg the answer sheets , will fcie^.pre'-pnnted 
v^ith all necessary student ^i'dentiffcatton information usind" 
a computer printing proces's describe.d Vaster under the data 
* processing section^ < . ' " ' ^. * 

\ " The testing schedule for each class will be reviewed ^ 
witti each classroom teacher and minor adjustments may be 
*made to accomrnodate' assemblies, special teachers or other 
unanticipated problems* Classroom teachers and'aides will 
be permitted to leave the classroom if they wish during the 
test administration period however this option will be dependent - 
on the approval of the principal. - „^ 

i i • Classroom Observations and Other fiata 
Collection^ Classroom observations will be conducted during 
the periods^ of October 1,1 through November 19, 1976 :and from s 
January 10 through.April 1, 1977 , a total of 18 weeks. Each 
classroom will be observed by j^ingle observer for a period 
of at least 3 hours ten times during the school year. Seven ' - * 

of these observations will be to assess program implementation 
variables and three to assess classroom climate Variables,- * 
The classroom climate visitation will include the full school 
day while the program^implementation observation will last 
about three hours including interview time with* the teacher, / • 

The regional liaison coordinators' will conduct several 
classroom observations with each observer to provide a check of 
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the observation reliability. The two person observations ' 
will be spread out to provide reliability checks over time 
as well as checks on the observations for- both program and 
cl,as,sroom climate variables: The prima iry assignment of the 
regional liaison coordinators will be to maintain contact 
with study sites and to! make sure that the^ detailed schedule 
of local training, testing, observation and data collection is 
followed.- The coordinator will contact schools before e^ch 
observation visitation to remind them of the' purpose of the 
visit and to check on the availability of needed personnel 
or resources'. The regional coordiliator will be in daily 
contact with classroom observers during the field observation 
phases for the purpose of identifying problems and confirming 
' upcoming -vis its.' Coordinators will arrange for rescheduling 
of visits by observers in the cases of school closing, teacher 
absence, etc. 

Regional coordinators will also be in charge of maintaining 
liaison and support of conmunity and teacher groups. This may ' 
involve making presentations at various meetings to explain 
the study and its benefits as they relate to the school and the 
community. Interviews with teachers, principals,. and selected - 
pareats will also be conducted by classroom .observers during , 
observation visits, usually during the afternoon. Data to be- 
gathered from school records, such as attendance data and parent 
occupations will also be gathered by the observers during non- 
observation time; 
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While every effort will be made to clear all scheduled 
visitation dates and to confirm eadh v^sit a week before it 
Is to occur, alternative activities will be scheduled in 
advance whenever possible in case there shou^ be cancellations 
These alternative activities will be observations of other 
study classrooips, collection of student record data, inter- 
views with principals, teacher aidl&s, etc. Howler, the 
probability of a successful resehe.duling will not be very 
great parti cular;ly where sites are spread out. The success 
of rescheduling will also diminish as the year goes on and 
a^s various alternate activities are completed. ^ 

Quality control monitors will work with classroom 
observation teams In order to solve problems that may crop up 
in the collection of observational data/ The monitors will 
have conducted the training of the observers and will hav^ 
developed a good deal of personal rapport With the observers. 
The observers will meet with the monitors in the centers as' 
a group about once fivqry two weeks to discuss in general t+ielr 
observational and other data col|^ction experiences. The 
quality control monitors will also periodically ^isit sites 
to check the reliability of the classroom observations. 

Complete travel and lodging arrangements will be made 
for field observation teams' and quafity control monitors as. 
part of the scheduling process. Living accommpdations will 
te made for field workers when an assignment to a study center 
necessitates relocation. Travel arrangements wrll be made 
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by the Secretary of the Field Services group with the 
assistance' of a travel agency- 
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5. Data Processing and Ana1y.sis Group 

The pritnary concern of the datar processing and analysis 

group will .be to provide accurate computer analyses of all ■ 

data collected during the study. The schedule of major tasks ^ 

to be completed ar^shown in Figure- IV, 15, The actual procedures 

to be completed have been discussed i^n'the data analysis section. 

The highest priority of the data processing analyses will be 

to complete the total, analysi s' as soon a,s possible after the . ^ ^ • 

completion of posttesting. To accomplish this goaVmany 

things will be done earlier in the study in preparation for ' 

• . * . .... 
the final analyses.' - . ^ ^ ^ 

During. the contract peViod prior to the 1976-77 school . 

year the planned data processing and analysis' plap will hfy -^'''^ 

implemented* and tested. Since it has been recommended. - - // 

that existing program packages ba used, no major programming' 

, will^ have .to be done. However, it will be necessary .to prepare- 

- and test sets of job control instructions whixh define, 

allocate^ and access appropriate computer files; tie together * 

various programs used in' the job. stream; and indicate the 

* various options to be used in the existing program packages. 

Some small .programs for. error checking, file creation, 

production of file verification lists, and pre-printing of 

student answer sheets will have to be prepared and tested. 

The data from the pilot test will be processed using the 

same techniques and time frame for the final analysis. While 
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the amount of data will be much less* thts will be a trial run 
with real posttest data. t» * 

Randomly generated data files of approximately the same 
size as the files for the total study will be create'd and 
run as a simulated test to make sure there are no difficulties 
in the computer -programs , printouts, or the computer processing 
job stream. Data files will be created and built during the 
school year with pretest scores, observational data, and 
interview data entered and verified. All files will be safe- 
guarded, using several back-up methods. Since all data will 
be collected on specially designed, optically read computer 
scoring sheets, there will be a minimum of coding and key 
.punching. There will be a need to visually scan- student 
answer sheets to make sure answer marks are properly darkened 
and that erasures are complete. This wjll be done by test 
administrators following the completion" of each- test in order 
to speed the processing of data at the central computer facility. 

During the pretesting, test administrators will enter a 
specially determined code to identify each answer sheet by 
school; class, and student. This will be checked against 
-class lists by verification clerks. Test ad(ni nistrators wi'll 
also fill out an, optically read student data sheet for each 
child for, the purpose of creating cpmputer files. 

Answer sheets-for posttesting will be prepared by computer 
with the student ' s^-name and proper identification, codes in 
order to facilitate the posttesting process and the rapid 
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collection of-%ta^ The^machine readable pfe-prtnted coding 
wtll be double checked.with class lists by verification 
clerks prior to distribution to test sites. The contractor 
win a] so take precautions to insure the darkness of the 
pre-printed codes in order to avoid unnecessary time loss in 
the processing of posttest data. 

The use of pre-prlnted answer sheets with student identify 
i cation coding will facilitate the processing of posttest 
data in several ways, (1) avoid errors in coding of student 
identification codes, (2) save time in the. actual testing 
process, (3) provide an immediate indicator of absentees to 
test administrators, and (4) permtt entry of data to begin 
prior to completion of absentee testing. 

During the screening and contracting period the data 
processing and analysis group will provide an interactive 
computerized information management system to assist in the 
screeninglmd selection process. The system will keep a 4 
'record of all contacts to prospective school sites by phone 
and letter, the basic information obtained fn the telephone 
and site screenings. By use of $he system it will be possible 
to develop summaries and listings of potential sites based 
upon screening data at any time or to determine which sites 
would be eliminated. This process will be very useful in 
determining the best possible sampling scheme from the pool 
of possible study sites. 



IV-47 



o ^ 173 

ERLC 



Figure iy.16 shows the organization of the Data Processing 
and Computer Analysis group. The Assistant Program Director 
for Data Processing and Analysis will have to be a person 
with, considerable experience^in the management of large 
educational computer data base files. The person should also 
be competent in the basics of data processing, computer 
progranniing, optical scanning, job'contVol languages, and 
use of Interactive management information systems • The 
person will be responsible for trainjing and supervisinghthe — 
data processing" and.verification' clerks, - 

Other data processing personnel- wiil be used o'nly on 
an as-needed basis during the processing of pilqt test data, 
pretest data, observation data, and post^^it data with ' 
-expected peaks during October of 1976 and March through May 
Qf 1977. 

Other staff needed will be a d^ta^ntry and file 
maintenance specialist. This person will be responsible 
for operation of the. optical scanning equipment loading of 
data into computer files, providing verification check lists, 
and performing file maintenance and back-up procedures. 

The verification clerks will c-heck over Incoming optical 
scanning sheets in preparation for the scanning procedure, 
perform random, comparisons of computer generated verification 
listings with original source documents, and maintain files 
of original source documents. They will also verify pre- 
printed answer sheets prior to posttesting and assemble 
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IV. 16. Organization oj^ Data Processing and Data Analysis Group 



. Assistant Program Director * 
for Data Processing & Analysis 



Ha-lf Time: March, 1976 to February, 1977 
Full time: March, 1977 to July, 1977 



Data Entry and Fil*e Maintenance 
Specialist 

April , 1976 to May, 1976. 
or October, 1976 to Oune, 
1977 "as needed" 



Data -Verification Clerks 
"as needed" 

April, 1976 to May, 1976; 
August, 1976; and October, 
mi to June, 1977 
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complete testing packages for distribution to testing 
administrators. A senior verification clerk -will edit .and 
luaintain control over all data documents, tape and disc ' 
files. This person will log in all test data as it' is 
received and maintain the system^ of safeguards. 
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6: Monitoring and QualUy Contt^ol Group 

I 

. * 

Xfhe major concerns in providing monitoring and quality 
control services, to the study is to make sure that the study, 
is being carried out according to the intended design and 
that data being collected is accurate. 

The Monitoring and Quality Control group will also 
work closely with the National Advisory Committee to solicit 
their criticism and suggestions regarding the implementation 
of the study, the procedures for conducting it, and 
'development of the final report. The Monitoring and 'Quality 
Control groue and the National Advisory Committee will be 
•responsible for clearing all study procedures with respect ' 
to. the protection of human Subjects. 

An Assistant Program DirectoV in charge of monitoring 
and quality control will head up the- group and" will report 
directly to the Project Director. Thi sjiersoQ^will period- 
ically review the study activitle^^it various chebk points 
according to pre-set criterion levels to determine if study 
activities are being completed on schedule and at a desired 
level of quality. .In the event that quality control checks 
reveal deficiencies, recommendations for changes in pro- 
cedures win be, made and considered by all project leader- 
ship personnel as a matter of hi^j)jfts4;^^*dor1ty. The tight 
time line of the study dictates that quality control 
assessment be attended to on a regular basis and concentrate 
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efforts on preventive measures, rather 1;^an remedial action; 

Some of the mora critical areas of Concern that will be • 
attended to during the start-up period are the areas of. 
training of test administrators and classroom observers. The 
Monitoring and Control group will arrange for external reviews 
*of the objectives, procedures, and materials that will be 
prepared for the training programs. The inter-rater 
reliability of the observation sta'ff will be assessed in 
actual or-^ simulated classroom settings as'part of the training 

• * ■ • 

The student testing procedures will be tried out and' modific- 
ations will be recommended as necessary* 

•"" During the actual study persons who had developed and 
conducted the training program will periodically meet with 
observation teams and test administrators'' to assess and share 
solution strategies to common problems that arise. Observ- 
ation and testing staff will be requested to make problem 
reports to the Quality Control group whenever unanticipated 
problems arise that ere not handled by standard procedures. 
These will be reviewed and if warranted discussed wi^th other, 
project staff in order to -determine a solution strategy- 

The Quality Control group will conduct some field 
operations but these will be limited primarily because of 
cost. 

Figure IV. 17 gives a schedule of major quality control 
tasks. Figure IV. 18 shows the organization of the group. 
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* Most of the staff functioning In. the capacity of -quality 
control will be assigned onVy part time and. should not, with 

' t'he exception of the trainers, have other major assignments 
to th.e project in order to provide a greater degree of - 
objectivity. 
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V '5, c . Xtgutti'-^.l?"^- Monitoring and Quality Control Organization 



Assistant Project Director 
• for MonJT-toring and Quality Control 



outside Expert 
Reviewers 



Classroom 
Observation 
•Monitors 
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IV. ' LOGISTICAL PLANS - 

B. LOGISTICAL PLANS FOR AREAS- OF PARTICULAR CONCERN 
* - I. External Advisory and Review Processes - 

There are two major purposes that should be considered 
for the establishment of" a National Advisory\nd Review * 
Conmittee. First such a connittee can provide expert 
criticism and advice with respect to the study design, in- 
strumentation, procedures, execution and analysis, and*" , 
f'eporting of results. The second reason is to establish ' 
that the contractors*, design, is. in compliance with 
Acceptable standards for the protection of human subjects., • 

Early in the study a group of nationally well-known ' 
arid respetted experts in diverse fields will be invited to 
serve as a National Advisory Committee to the study. Their 
first task will'be to r'eview the study! desiign, study procedure 
and test instruments with-respect tp the possible fisks-to 
students involved in the study. Committee reviews will be 
conducted with objectivity and in a manner to ensure the 
exercise of independent judgment *of the members. Records of 
^all committee reviews will be maintaine^d and made available 
to participating school districts as a means of providing - 
assurances regarding the study. - ' 

The committee will be concerned with the possibility 
of risk: physical, p^chological , spdolomcal , or other 
as a consequence of any activity which i^^associated with the 
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operation of the Study. It is the purpose of the Committee 

to analyze each planned activity to determine that: 

• the rights and welfare of subjects ^ 
are adequately protected 

f the risks to subjects are outweighed* 
.by the potentiaTbenefi ts 

t the informed consent .of subjects will 
be obtained by methods that are 
adequate and appropriate ' . 

During the -in-iti a 1 review process of the study design, 

procedures, and. instruments the experts will also reflect on 

the adequacy of each in attaining the objectives of the study. 

Should written* reviews by the Committee call for any major 

modifications of the design, instruments or procedures, 

such changes will be considered imnediately by, the Project 

Director and the Research' and Implementation committee. A 

response and recommendation will be prepared and reviewed 

by the Committee and the NIE project offices. -Upon the 

agreemer\t of 'the Cbmmi ttee.and the NIE the changes will be 

made. ^ Minor re commend actions for change will be reviewed ^ 

by the Research and Implementation group and necessary 

•action taken subject to. approval of the Project Director. 

2. Incentives 
» 

One of the most pressing logistical problems in conducting 
the study will be securing the agreement^ of school districts 
to-participate. This will entalT obtaining, the support and , 
cooperatiWi. of the school principal, teachers, district ^ 
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administration^ school board> school /community groups > teacher 
organization, and parents. In addition to screening sites, 
study staff will have a large public relations job to con- 
vince districts of the benefits of participating in the study. 
It is anticipated that some. districts will be positively 
influenced by the prestige associated with participating in a . 
large national study, particularly where a district is 
proud of its accomplishments. However it is expected that 
while this may be an initial motivation for .many districts, 
it will not be a lasting factor in very many cases. 

In order to overcome the possible resistance, the 
contractor Will have to convince the chief school district 
administrator of the value of the study and also provide, 
assurances that participation will riot reflect poorly on the 
schools or the community! Therefore in consid6r1jrg Incentives, 
the contractor must not only consider pos-itive contributions ' 
but avoidance of negative consequences. In this regard, things 
that will be helpful would be efforts for securing the cooper- 
ation of community and teacher groups, assurances that all tes^s 
and procedures have been designed to protect the rights of 
students, assurances that proper safeguards will -.be taken'ta 
preserve the anonymity of student data, and assurances that 
children will not be harmed in any way by the testing, observ- 
ation, or interview procedures. Assurances .will also havet to 
bejgiven that the contractor will in no instance include specif i 
' mention or .identification of the school district, schools, or 
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teachers In any reports prepared In association wUFi the 
project, including daia-which will be.jkfAlKUwi^. 

Some of the positive incentives that would be used would 
be payments to district personnel for time spent in connection 
with the study; providing all test results to school districts; 
and providing presentations about the study upon request to 
PTA's, teacher groups, etc. 

Since the study will use local teachers from the sub- 
stitute teacher lists to act as test administrators; there 
are several benefits to the district which may be yiewed as 
incentives. First substitute teachers would be getting 
training in test administration that will enhance their 
use'fullness to the district. Secondly, since most districts 
like to maintain good relations with their substitute teachers, 
the added time and pay for working as a test administrator 
. during periods of time when teacher absence is low (September 
and April) wilfJbe appreciated and help the districts build 
better relationships with their. substitute- staffs. 

The folloV/ing is a 'list of the substantive incentives 

that would accrue to districts participating in the study: 

(a) The use of, school dis^trict substitute 
' teachers as test administrators during 
usually slack teacher absence periods 
will provide additional pay and good 
will among substitute teacher staffs. 

' (b) The trai^^ing and experience of sub- 
stitute teachers in test administration 
by t>ie study will provide a -new set 
of skills that may be useful to school . 
districts. 
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(c) Raiments to teachers and principals 
and other district staffs* for th«^r 
time making specific contributions 
to the study; i.e., interviews, . 
testing, recording- student and in- 

S structional data. 

(d) A copy of the final study report. . 

3. Communi t^*>lDwl vement 

It should be considered essentia] to secure the cooper- 
ation and rapport of par^ents and school community groups 
for operating this stujdy in view of the rather intensive 
testing-, observations and interviews that will be conducted. 
Essentially, parents and conmunity people'^ill hav'e to be 
assured that the study will not harm or adversely affect 
the progress of their children, and, in fact, may produce a 
positive contribution to their well-being by providing 
information useful for progrkm improvement. 

During the early stage of the screening and contracting 
process, principals and central office administrators will 
be asked to provide information on local parent and conmunity/ 
school groups, the leaders of these groups will be contacted 
later in the screening process by letter and with a follow-up 
telephone call to c|i;scuss obtaining theic support. Brochures 
explaining the study will accompany the initial letter. 
Since conmunities are expected to be very different it is 
anticipated. that the nature of the reactidfi to these 
contacts will be highly variable. . ' , 

i 
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The primary points of the contacts with local parent 
and community school groups will be: (1) that the -group was 
contacted, (i) that the study has been described, (3) that 
limited numbers of parents will be interviewed as part of 
the study, (4) that. the study will not be harmful Ind may 
^help to bring about improvement in the district; When it 
seems appropriate representatives of the contractor will 
offer to make a presentation about the study to interested 
school or community groups. 

: 4. Professional Support ' 

A potentials difficult problem to overcome with respect 
to- the study wouldXbe obtai.nipg and maintaining the support 
of professional education organizations in the participating 
districts. To this enci, the contractor wi 11 need to provide 
information about the study to the district or school repre- ' 
spntatives of the teacher organization during the spring 
of 1976 as part of the screening visitations. This information 
will describe the intentions' of the proposed study with 
.specific requirements and benefits to teachers. The contractor 
, would also solicit- comments and suggestions with respect to 
the' study as it affects' professional and supports instructional 
staffs. Study personnel will contact these representatives to 
follow up the initial contact and Solicitation. Should ^ 
' serious problems appear with regard to accommodating * the desires 
and requests of -the- professional group or union,^ thejuse of the> 
district or School would have to be reconsidered in^^iew of the 
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possible adverse effects. 

During the school year of the study, it wduld be 
suggested that the arta regional coordinators in the Field 



Operations ^up maintain positive contacts witli regard 



to the study and its progress in order to h^ad off any 
possible problems before they develop to a union grievance 



5. Confidentiality of Data 

One of the most sensitive areas involved in conducting 
evaluations of educational programs pertains to the confi- 



by S. J. Res. 40, indicates that no funds will be made 

available to any educational institution: 

"... which has a policy or practice of permitting 
the release of educational records... of 
students without the written consent of their 
•parents to any individual, agency, or oraaniz- 
ation other than to the following... (6F) 
organizations conducting studies for, or on 
behalf of educational agencies or institutions 
for the purpose of developing, validating, or 
administering predictive tests, improviii^ 
instruction, if such studies are conducted in 
such a manner as will not permit the personal 
identification, of students and their parents 
by persons other than representatives of such 
organizations and such information will be 
destroyed when no longer needed for the pur- 
pose for which it was conducted..." 

The procedures of the study will attend to the concerns 



for privacy of student and parent data by maintaining data in 



to 

6 




stage. 
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such a manner. as not to permit personal^ Identification of 
students and their parents by other than representatives of 
the contractor. This will be done through use of a student 
numbering system for the data files. In order that post- 
test data be Identified with matching student' pretest data, . 
a separate- student name/numb.er file wIlT be maljitalned* [)y 
the Assistant Program Director for Data Processing to be 
used'fdKthe creation of posttest answer sheets. Following 
the creation of the answer sheets, the file will be des- 
troyed as will all answer sheets and data collection forms ■ 
following the optical. scanning process. In the process of 
archiving data from the study, all identifiers such as names 
of schools, districts, teachers, principals, observers, and 
students, etc/ will be removed. While this will not prohibit^ 
further analysis, it will protect individuals from iden-y^fic- 
ation in future reports. 

While it is felt that^these pif*ocedures meet. the sp-ifit 
and specification "of this Buckly Amendments, , at this present time 
no legal decision has been made to clarify the situation: |f 
a participating school district did not feel that these pro- 
cedures were sufficient and would prefer that paren.tal 
permissions be obtained, in view of the unsettled legal qqestiops 
the contractor would be obliged to collect parental permissions. 

The procedures for collecting the parental, permission \^uld 
be rather simple. The contractor would prepare a letter t|1v1ng 
a short description of the study along with a parental permission 
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slip. These wQ(uld be printed at the coi^tractor's expense 
by the school/district and distributed to all children 1n 
classes to be studied. The permission slips would have 
'to b^ returned bef dre the start of testing on September 20 
1976. 
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IV. . LOGISTICAL PLANS 

C. MANAGEMENT/STAFFING/SCHEDULING 

This section presents an overall view of the study fT:om 
the point of view of task schedules for each of the work 
groups as well as the .scheduling of staff to perform these 
tasks. At this time the specific assignment and scheduling 
of tasks has not been brought to the level of detail where 
specific tasks and subtasks may be assigned to specific 
positions. This detail will best be completed when specific 
indiv-iduals who will be assigned have been identified-. 
This should occur in the proposal development stage. 

Generally, because of the changing nature of the pro- 
posed project between the first seven months and the final 
ten months, the staffing plan attempts to make use of staff 
over the entire project. This necessitates moving personnel 
from assignment to assignment several times during the study. 
However the plans indicate that this concept is not entirely 
feasible and it appears that some staff positions do not 
naturally feed into succeeding tasks of positions. "It should . 
be recognized that this will create a surplus of highly 
qualified and highly paid staff at the end of the screening 
and contracting phase. When possible, these similar positions 
have been accommodated in the Field Operations group in order ' 
to provide continuity; however, the contractor will. -very possibly 
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have to shift some people to other projects outside this 
study at the beginning of the field work phase. Figures 
IV. 19 and IV. 20 show the overall schedule of study tasks 
and personnel. 
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IV. LOGISTICAL PLANS 
b. FACILITIES 

The facilities necessary to perform the proposed study 
consist of the following basic needs: 

• Office Space and Equipment 

• Wide Area Telephone Service Telephone System 

• Optical Scanning Facilities • 

• Computer Facilities 

• Printing/Reproduction Facilities 

• Training Facilities 

• Video Recording and Playback Equipment 

This section discusses the basic facilities needs for 
the study with suggested means of providing than. 
1. Office Space and Equipment 

The need for office space is generally based upon the 
number of full-time employees assigned to a project. In this 
casei due to the extensive number of staff which will be working 
in the field for extended periods of time, there is a reduced 
requirement for office space during the school year. It is 
anticipated that during the first seven months of the project 
that about 22 staff personnel will.be working rfequtring abeut 2100 
square feet while during the school year only-T500 square. feet ■ - 
will -be needed. • ^ 

The estimated computations for office space are as follows: 

February to August: 2700 sq. ft. x $9.24 per sq. ft. " 

X 7/12 year = $14,553 

September to July : 1200 sq. ft. x $9.24 per sq. ft. 
' X 13/1 2 year = $12.Q12 

' TOTAL RENT. . $26,565 
' . . IV-72 ' . . 
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In addition to floor space the contractor will need 
standard office equipment including desks, chairs^ tables, 
typewriters, bookcases and cabinets.. 

2. Wfde Area Telephone Service Telephorte System 

Because of the heavy emphasis on the use of the telephone 
for screening purposes and because the proximity of proposed 
sites cfovers the United States east of the Mississippi River, 
it seems that the contractor should give* serious cqnsideration 
to Wide Area Telephone Service to reduce^ the cost of the 
screening and other field operations. To this end Figure 17,21 
contains an example breakdown of a recommended terephonip 
system designed to provide for heavy use during February and 
March of 1976. This service vill t+^en be rtduced to one intra- 
state and one within-state line for the remainder of the 
study. It is also estimated that the contracting corporation 
would arrange to share the phone line and cost for concurretit 
projects. 

For purposes of comparison, it is estimated that during 
February and March 2000 ^long distance screening phone calls 
will be made at an average cost of $5,50 per call , a total of 
$11 ,,000, The 'comparable cost for a shared WATS system during 
•February and March 'would be $6,426. 

It should be pointed out to the NIE that if the contractor 
does not have WATS service, an orjJer for such service will have 
to be placed with the telephone company by December 1 , 1975 in 
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order to have the WATS system installed in time for its 
major use, the screening. 

3. Optical Scanning 

Due to the heavy volume of test scoring and other data 
collection which must be accomplished during tlie study, 
use of an. automated test scoring system seems essential. For 
this reason the contractor shduld have access to an optical 
test scoring device such as an Op Scan Model 100 DM System 
^which has a scanner and magnetic tape unit. The contractor . 
should also -provide for back-up scanning capability should 
the primary facilities be inoperable during the peak post- 
testing period in the last week of April' and first week of 
May, 1977. 

4. Computer Facilities 

The. computer capabilities recommended for the program 
would be' best provided by a large scale multi -processing 
system such as an IBM System 370 Model, 168 or Control Data 
Model 6800. Such a system should have a complete set of 
"social science research software packages such as UMAVAC, 
SPSS, BMD, SOUPAC, etc. The system should also provide large 
on-i;ne files as well as "tape drives, disk packs, and high 
speed p -inter all of which are standard at most large 
research computer centers. Shou'ld the contractor not have 
this' installation on premises, a remote keyboard terminal 
with high speed printing capability should be considered. 

IV -76 
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5. Printing and Reproduction « 

The contractor shquld have access to duplication and 
printing facilities to produce the various brochures, test 
books, training materials, and reports required for the study. 

6. Training Facilities \ 

At several times during the study, the contractor will 
need to provide training to large and small groups. Because 
of the relatively short duration of the training period for 
large groups, the screening consultants (3 days) and the test 
administrators (1/2 day In the fall and 1/2 day In the 
spring), temporary arrangements can be made for large meeting 
facilities. The contractor should be able to conduct other 
trjalning In, medium sized meeting rooms, • 

i 

7. Video Recording arid Playback Facilities 

For the purposes of training study staff in the skills 
of Interviewing, classroom observation and screening a small 
black and white video system will be highly desirable. Such 
a system should be portable in order to make the classroom, 
tapes and also be capable of playing tapes made on other 
standard video systems. 
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IV.. LOGISTICAL PLANS ' 
• . i. FINANCIAL PLAN 

This section contains general cost estimates for 
'conducting the proposed Study. The pricing has not been 
, done at a detailed level on some items because of the minor 
nature of their contribution to the total fiost of the study. 
The costs which have been estimated are adjusted in terms 
of inflation for what one might reasonably expect to pay 
during -the actual contract -period. - Thi s assures a . . 
moderate rate of inflation of around B% per year; a rate 
based upon the increase in the National Consumer Price Index 
for all cpnsumer items over the past six months reported by 
the United- States Department of Labor in Monthly Labor 
Developments , September, 1975. 

The parameters for the pricing are explained as the. 
rationale for the budget sumtiary appearing 1n Figure 17.22. 
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figure IV. 22. Budget Summary 



Consultants 


$ 904»395 


iiaVci aiiu Luuy 1 iiv| « 


208 040 


Telephone 

^^^^^^ 

Rent 

V _ 

Computer Processing « 


30»000 


^ 26»565 


15»000 


Ootlcal Scannino Forws— 
and Scanning Costs 


8»000 


Misceflaneous (23; of $663»000 
Basic Salary Cost) 


, 13»000 


Printing 


10»OQO 


Supplies (1.43; of Basic Salary. Cost) . 


9.000 
'1»224»000 


Overhead (18.43; of Total) 276»000^ 




$ 1»500»000 
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Costing Parameters 



A. Classes to be studied in depth 



1. 



Number of classes 



74 standard-sized classes 



30 large or open-classroom ^ 

2. Location ( 

Assume all classes will be located in a minimum 
of 20 districts located in or near urban centers 
east of the Mississippi River. 

3. Testing 

I 

(a) All testing will be conducted by a Test 
Attalnistrator and the classroom teachers. 

(b) Test Administrators will be trained and 
paid by the contractor. 

(c) Testing will be spread out in time so 
that children will not experience long 
testing sessions (longer than one hour •. 

\ without a break). 



(d) Make-up testing sessions for absentees will 
be conducted by the Test Administrator who 
will work several half days following the 
testing for this purpose. 

(e) Two. Test Administrators will be used for 
large or open education settings with at 
least two other teachers' present. 

(f) Pre-testing ; Five hours of testing per 

, class wTIi be administered over 2 days during the 
Ust two weeks of Septemben (September 20 
to October J, 1976) with provision for four 
half .days for make-up tests. 

(g) Post-testing ; 

(1) Cri teri on~Ref erenced tests : Two hours of 
testing per class administered during the 
last week of March or first week of April 
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(March 20 to April 17) depending ' 
on scheduling of spring vacation. 

(2). Achievement and affective measures ; 
'Four hours of testing per class 
administered during the last two r 
weeks of April (April ^18-29) depending 
on scheduling of spring vacation. 

4. Classroom Observations 



(4) Program Implementation : Six class 
visits spread evenly during school 
year (September 20 to March 19, 1976- 
and January 10 to April 1, 1977). 
One observation will be done by two 
observers. Observations of large 
or open classrooms by two observers 
and a classroom observer and an area 
coordinator. (Observers will ob- 
serve an average of four days per 
week with one day per week for making 
arrangements, conducting interviews, 
or travel.) 

(b) Classroom Climate : Three class visits 
during the period of January through 
March, 1977. One visit will be done 
by two observers, a classroom observer 
and an ?irea coordinator.. Observation 
for classroom climate will last an 
- entire school day. 

' B. Staff Requirements ' 

1. Classroom observers 



(a) Number of observations— 1206 

(])) Available school observation time— 18 weeks 

(c) Average number of observations 

per week— > 4 

(d) Number of observers needed— , 17 

(1206 1 18 7 4 = 16.75) 
$1200 per mo. &3«rage Sal. & Benefits 
\ for 11 mos.— ' $224,400 
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(e) Alternate classrtfom observers 
3 observers for 2 months 
at $1200 per /month 



TOTAL CLASSROOM OBSERVERS 



$ 7.200 
$ 231,600 



2. Area Coordinator 

> (a) .Number of observations— 208 

Cb) Available school observation time— 18 weeks 

(c) Average number of observations ' 

per week-- .» 2.2 

- (d) Number of area coordinators— 4 
208 7 18 4 3.0 4 3.85 
$1983 per mo. average Sal . & Benefits 
■for 12 mos.- $ 95,200 

3. test Administrators 

(a) Pretesting Study Classes (?720 - 10/1) 

■ (1) 37 Test Administrators for staridard 
Classes; 11 half days ^ $22.50 per 
half day (1/2 day. training, 5 half 
days testing, 5 half days; for make- • 
up tests)— " • 

$ 9,157.50 

(2) 60 Test Administrators for open 
classes: 8 half days 0 $22.50 
per half day (1/2 day training, 
5 half days testing-, 3 half days 
for make-up tests)- 10,800.00 

(b) Posttesting Study Classes: 37 Test . 
I . Administrators for standard classes. 

work 13 half days @ $22.50,- 10,822.50 • 

60 Test Administrators for open 

classes-work 10 half days @ $22.50- 13,500.00 

TOTAL TEST ADMINISTRATORS ^ $ 44,280.00 

Q. Travel Rfequirements Field Operations 

1. I^ental of 17 compact cars 0 $42O per mo. ' . 

for sevfen months- $., 49^,980. 00 
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2. Gas Cos tjs:' 2000 miles per mo. for 
7 mos. 9 20 MPG 0 $.70 per gal. for 
.17 cars— $ 8»330 

'* ' • ' . ' - 

3. Other' car rental for Area Coordinators - . 

300 days 9 $75 per day— • 22»500 

4. Lodglng.per diem 

(a) 118 idays In central location for 
training of 17 observers 0 $25 

per day— - - , - 50,150 

(b) 2 days per week for 22 weeks 

9 $35 for 17 observers during . ■ 

SQhool observations 26,180 

(c) Area Coordinator lodgi-ng 40 days 

e $35 for- 4 coordinators 8,600 

5. Travel to and from study Centers 

17 observefs':-8 trips I $150— 20,400 
4 coordinators: 10 trips '(? $150— 6,000 

6. Test Administrator travel for training 

2 trips 9 $2? for 75 observers " 3,750 

7. Travel for screening consultants (300 

visits at $25 each) . . . ' - 7,500 

8. Travel and lodging for training Screening 
• Consultants (30 trips x $150 + 30 x 

3 days x 



TOTAL TRAVEL AND LODGING 



7,650 
' $ 208,040 
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'iVr LOGISTICAL PLANS • 
, F. . REPORTING ' 



. Because of the'ttght time line of , the proposed study, 
there will -be a need to Iceep^^the reporting process as 
efficient as possible. For this reason the proposed interim 
or milestone reporting formats are designed to be. as brief 
and ir\expens,ive as possible." Figure IV. 2'3* contains a. -list 
of proposed mil es^tone reports. ^ * 
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Figure IV. 23. Proposed Milestone Report 
Page 1 of 2 



DATE 



3/30/76 



.MILESTONE. 

■A 



-DESCRIPTION 

(1) List of programs "having • 
claims for' providing in- 
dividualized reading and/ 
- or math for elementary grades 
(K-6) • c;^ ; 

. (21 Li sti ng of programs qual i fyi ng 
, as individualized on the basis 
•of initial screening with 
summarized results of screening 

(3) Summary of recruiting activities 
to date 



6/30/76 



V 



10/30/76- 
1/30/77 



A 



A 
A 



(1) Sunimarized resuHs'of pilot 
test ' 

(2) Training program 'docpents 

(3) Summary \i recruiting and ' 
staffing activities 

• • * * 

(4) List of nominated schoaVsites^ 
wtth results of screening 

' indicated 

(5) -Validated observation iiistru-\ 
mentsand procedures", interview 
schedules, tests and test time 
procedures - . 

[\) Listing of participating 
schools » . - 

(2) Sunmarized results of pretests 

Summary^ of Fall' ifhplementation 
obs6r vat tons and "data collection 

Summary of Spring classroom climate 
and implementation observation data, 
collection 
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DATE 



6/30/77 
7/30/77 . 



MILESTONE 



A 
A 



DESCRIPTION 



/ 



■Draft of Final Report 

(1) Final Report • 

(2) Compute^^ed data- tap6s ^ 
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^ A. INSTRUMENT AND DATA CLASSIFICATION SCHEME 

1, Study Question. 1 

2, Study Question^'Z 
^ 3, Study Question 3 

4, Study Question 4 
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B. INSTRUMENTATION 

1. Forms for Sampling and Selection 

School Dj strict Interview Form (SDIF) , ' ^ 

School Interview Form-Traditional Site (SIF-TS) 

School Interview Form-Indivi^Wualized Site (SIF-IS) 

Classroom Environment-Screening (CE-S) 

(Rationale for Suggested Classroom Screening Variables) 

Teacher Questional re-Screening (TQ-S) 
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SCHOOL DI^STRICT INTERVIEW FORM (SDIF) 



District _^ \_ 

^ . ^ 

Address ; ' 

State 

Telephone # - - 



Interviewer Date of Initial Call / / 

Persons Contacted . Telephone # (if different) 

Name ' Position (Date Called) 



Contact Re 1/ Program 

Inforhiation Source 



Hello. V\y name is . I work for . 

. contractor's name 

in^ . We are planning a comparative study of 

contractor's location 
individualized and traditional instructional programs under a contract with the 

National Institute of Education. The actual study will be condu^cted in about 

100 classrooms nationwide during the 1976 - 77 school year. I have coritacted 

you in order to ask for your help in identifying potential sites for the study. 

The study is one of several being sponsored by the National Institute of Education 

to provide information to the Congress for the consideration of new compensatory 

education legislation, A number of program developers, state and federal agencies 

have already cooperated with us irj identifying individualized programs that are 

• being used iK compensatory education settings and are worthy pf study. ^ 

O V-8 

ERLC , • 



program name 



School District Interview Form (2) 

Your district has been identified as one that uses the 
program. 

Question 1: l6 tivU Inionmation coHAtct? . Yes . | | No 
If yes , go to question ffz! 

If no_, probe to determine what is wrong about the Information, 
e.g., the program has been discontinued, wrong program information, 
wrong district name, etc. If a simple correction is possible, 
continue with question #2. If the district does not use the 
named program or some other individualized program, apologize ^ 
and discontinue call. 



Question 2: How -many 6chooU in thz dUtMct luz thz 



Since the study we are planning deals with*the use of indi v.idualized 
programs in compensatory education settings, our criteria for the selection 
of potential sites requires that a school receive ESEA Title I funds. 



Question 3: Me any oi thz hckooU thcut luz 



phJOQKm M 



youA dUtAlct Tlttz I ^ckooU? 

Yes 



pKOQKm name 



No 



J 



OR 



I4 thz 6chbol that a6t6 thz 
Tittz I 6c}iool? 



pKOQKam a 



piocihm name 



Yes 



No 




If no, probe to see if schools qualtfy for "compensatory education 
funSr. If so, treat as Title I, if not, conclude conversation. 

If yes , continue. 
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Schoo>Di strict Interview Form ^(3) 



Question 4: tkt Titla I schools that a/vi u^ing th^ pn^oQnam, /lOw; moht/ 
cue tht pKog^m cut the. ioiVUh qnadt ZzvoZ? 



Question 5: A/ii thoAt TJJJiz I 6chooJU in youK didt/iicU that oie tAodlUonaZ^ 
gxoixp piiced iyvstmiction thaZ muZd be con^idoAzd a good 



Yes 



No 



If n£, probe to make sure before concluding the survey. 
If yes , .continue. 



The next phase of our' identification process requires that we contact 
principals oi^otential s,tudy schools. We have a number of questions 
, regarding the Reading and Mathematics programs in their particular school. I 
would like to call the schools in your district that we have been talking 
-about. This contact is only an initial step in our screejiing process and 
before we take any subsequent steps in thef identification process, should . 
the schools qualify, we would get back to you. Can you give me^ names of^ 

nncipal(s) in your .district that I should contact?' I would also need 
to have the name of the school and its telephone number. 




Principal 



Phone i 



School 



Individual ized 
or Tradi tioa^l 



Thank you for your time. It is greatly appreciated. We will be getting, 
back to you following our contacts 'to the schools you have mentioned. 
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SCHOOL- INTERVIEW FORM-TRADITIONAL SITE (SI-F-TS)' 



Principal 's Name 

School Name 

Address 

City 1 



Telephone # 



State 



: Zip 



District Name 



District. Office Contact 
Nominated Site Category 
Interviewer 



Traditional 



Date of Interview* 



Hello my name is 



I work for 



(contractor's name) 



~ (contractor's location) ^ \^ 

We are planning a study, for the National Institute of Education to study the 

effectiveness of various reading and majthematics programs^ that are being used 

in elementary schools receiving Title I support. The study which will be 

conducted during the 1976-77 school year will involve abput 100 sites nationwide. 

We are currently involved in- the initial screening process, for site selection. 

I have spoken with • at your 

(name of district office contact) 
district's central office and he/she indicated that your school is one that we 

might cqnsider as a potential study site. As part of our identifji cation process 

we have developed a short telephon? questionnaire that will help us determine if 

your' school meets the basic criteria for the planned study. 
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School Interview Form-Traditional Site (2) ^ ' " ' 

If .interviewer .notes resistance to questioning at any time in th^ 
interview, add this question: 16 thoAz motHvi.Umt that muJid be moKz 
dOYivZYiitnt OK l6 ikviz 6omzonz zlht Jiji f/ouK 6thool that might hoZp u6? 
(Take appropriate action,) ^ 

Question 1 : Vvi&t atZ m need to know appioxmatoZy thz numbvi 

diUidKm oXtQjfuUytg yoiui ^cJwoZ? 

^Question '2: ^hat QHadt IZ'^zlU^ ctoe4 youA ^diool 4eA,ve? _j 



If^grade 4 is not included conclude telesurvey. 
Question 3: U yovJi school an ESEk^TWiz I Sakoot? 

. . □'Ves ■ ■ . ■■ • 

[ j 'No If no probe --to be certain before concluding survey. 
Question 4: (^hat pzKdtntagz ol tht ckiZckzn in youA school qdati^y {^OK 

* TitZa I ivLYidad pioQnam ? ? ^% - 

Question 5: Vo ,ckUjdAm uko quaLL^y ijo/f. TiXlz I pnx)gHam Kzcetvo^-Kzading and 
math ini>t/uiction in tkt i^amt oJiohbU a& Yion-Tlttz^ I ^tado^nt^l 

|~] No^ If no, skip to question #8. 
Question 6: Vo Titlz I ckiZd/ien Ktcetvz additlonaZ inttmction in KdjadinqV 

V If yes, ask the following: P£.ea6e ducJitbe. thz natuAz thz 
additional i/Jo^k? it individual tatontng, 6maZl gnowp mhk, 
KQjxdtng Jbbonatonjy , etc. ' . . ' ' 
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S'chobl Interview Form-Traditional' Site (3) • ' , 
Question 7: T^e I chUdk^n Kzczlvz taxlcUJUj)}^^ iM^tmtitLon Jji math? 

' If yesVask the followV^g: ?i^e^<lucJuh& tkz. na{u/Le..o^ > 
tko, adjdWjOYiaJi mnk^ ^ It ' JbuUvldii^ tu±(LH^, mait'QKowp 



Qyestjon S.:' The stu^ly we are pjanning will deal only, with childrefl at 

the fourth/grade leveK, The- rfest of the questions, will only 
. / have to do vftth your program for fourth grade -Tevei children. 

^ i^hcut (Viz tkz mmu thz ba&lt taxU ok pho^Kjcm thcut you ' 
\ ' (Viz u^lng at tkz^ iquJitk ^nadz tzvuZ in matk?- ^ 



Xn Kmding ' 



. ^ Question 9: A/iz youn ^oufUh gnxidz qZm^q;^ gnjoupzd by abltcty oK-cuii thty 

F] by ability , ■ ' " ' /\ ' ^ ' ' 

heterogeneous " . 

Qtjestion 10: J4 ;C^e rmtk auAAtcuJim tuzd In you/i {^ouAth gKada IzvoJi 4pecc<J^ed 
in ttnxn6 .b(( bokavio/tdl objtctive^? / ' ^ ' 



-Question 11: l^^thz kqaj^^ cjwUcuZim 6pzcl^izd in Wtm o^.bzhavio^ijcit • 

/ • . Q'"^ • , 225 
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School Interview Form-Traditional Sit» (4) 

Question 12: Vo chUcUzn gzn^/iaZZtf. b^ln t/ooKk cut tht bzginyung thz lokool 
• yojoA out diiioAZifit pJUce^ In thz mcuth cjuJuUmlUml 

• Question 13: U Kidding? ~ . r - • 

Question 14: Voe^. thz tnatk pKOQnm makz pncvJUloM ioK AJidlviduh^ pacing?' 

Question 15: P^w y^iuA^KmLiYiQ pAjog/m? : 

• • - ■ . . ■■■ /. -. ; 

^ Question 16: (a)' Vo chUjiUejfi gznwaliy bojgln, a nm math topic at thz 

. ^ ^ ^ome -tune? , \ - ' . 

(b) Hota do Xf^achoJU decXxU.tkat thz puplt& a/iz Ktady to 
r " - . bzglYi a new math toptjd? ^ 



Question 17: (a) Vo chUxheji gzYivuxily bzgpi now rmtoJuAl in KOAding at 

tht lamt timz? \ ' ^ , ^ 

•■ . Lives • / ■ 

(b) How do tQjxchz/Uf dzcudz that thz pmpXJU' oKi KomLy to go 
on to nm mateJuaZ? - * ' ^ 



School Interview Form-Traditional Site (5) 

Thank, you for^pur help- and responding to this questionnaire. 
After we fiave reviewed the results of our initial screening process, 
I we will be in touch with 



district office contact - . - 
.jyjth regard -to fhe, selection-ot pos§ible*' study sit^s. We hope that 

you woLild be j^ierested in participating in the study. A^ain, 

thank you-fbr your time. , 
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SCHOOL INTERVIEW FORM-INDIVrOUALIZED SITE .(SIF-IS) 



Prtncipal 's Name. 
School Name . , 



Telephone #• 



Address 



State* 



Zip 



BUtnct Name 



District Office Contact 
Nominated Site Category 
Ijnferviewer 



Ihdivi dualized 



Date of Interview 



Hellos\ my name is 



I work for 



contractor s^ name 

in . We are planning a *study for the ^National 

contractor's location 
Institute of Education to study the effectiveness of individualized reading 

and mathematics programs that are being used in elementary school receiving 

Title Tsupport. The' study which will be< conducted "during the 1976 - 77 school - 

year will involve about 100 sites nationwide. We are'currently involved ir\, 

the initial screening process, for site selection-. I have spoken with ^ 

' and he/she indicated that your school is 

name. of district office contact 

one that we might consider as a potential study site., As a part of our 
indent^cation process > we have developed a short telephone questiojinaire 
that will help us ^determine if your school meets the tasic criteria for the 
planned study. * • 

If interviewer notes resistance to questioning"^ at any time in the interview, 
add this question: I-i theXz anothoA timz that would be mn.z zomzniznt on 
thcAz iomzom zJUz 'in you^ i'chooZ'Ifi^lUgfi^ hoZp a*? (Take appropriate 



action.) 
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ScKoorl Interview Form-Individual ized Site. (2) • - 

Your school has been identified as one that us€S the 
program. - ' " . ' 



No 



□ • 



'Question 1: U tkU inf^onmcuUon coAAtct? Yes 

If no, |>robe to be certain before concluding telesurvey. 
If yes » continue.. - . 

Question 2: ^ We woittd IZkt to knoio hoio many yza/Ui you havz been using 5t/ie 
. ^ - pA^ognxm? . years 

At whcLt QH.adz tzvzi^ do you-iuz thz p/iog/tm? - 



If grade 4 is not included, conclude telesurvey. 
Ask question #3 for IGE, I/D/E/A, PLAN or DISTAR,. 




U thz 




Math only 
Reading only 
Both 



j_pn,OQKm {16 zd ^qh. math on. Kzading on. both? 



tion 4:^ Me thoAt any othoA indlvAAuaZlzzd mading on: math pnognjom tn 
iUz at youA school? ' ^ . . 

^Yes ' If' yes , (joha^ oJiz^ thz namz6 OjJ othzA pnognm6 and 
at tohat ^nadz IzvtU oAz thzy luzd? (Take notes. 
Program Grade Levels 



No 



„If no, go on to the next item 



i 



Question 5: What i& thz appnoxAjnatz mmbzn. o{^ cklZdnzn attmding youA 
♦ ' •> 

school? • ' • 
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School Interview Form- Individualized Site (3) 
Question 6: l4' j/oua 6qhool an ESEAJ-Uli I Sekooi? 

Yes 



, If no, probe to be ceHain before concluding survey* 



Question 7: bJhat rpoAco^ntago. the. ckUdKtn aji yoiui school qaati^y ia^ T^itl^ I 



% 



Question 8; U the.\ 



paognm uied uxLth compo^Mcuto^y 



Qjduca tion ^ckUd^en cut the. ^u/ith gnxud^ le^vtl? 

zi ^« ■ ■• • ■ • ' ■ 

Question 9: 9o chxldUitn -voko qualify ^oK TMe. J^P^ogmk ^ececue ^ojidLng 
\ , ^ * and math ^tAuction ui the. 4ame cJta66U a6 non-TitZe. 7 
6tudznt6? ' ^ ^ ' ' 

No If n£, skip to question #43. ' ^ - 



□ 



Question 10: Vo T JLtle. I dkUxi/L&n fLZczive. addcUonaZ initmicJU-on Kexid&ig? 

Yes ; ' ' ■ . 

No ; * . . ' . ' 

If yes," ask the f^lowing: Plza(>e.-d&i>cnA,b& tha natuAz 
the. oddLtionaZ: mik; -i.e.. , -LndivA-duaZ tutoninQ , maZl gfLoup 
mnk, nzadinQ ia.bofLatofLy/(2Jx.{ 
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School Interview Form-IndijVidualized ^i'te (4) •' ' . ' 

Question 11: J)o T- UZz I ckUdfian xzciUve. cddAXionaX i*i^.tAucJU.on in math? 

Yes 

I • . 

No 



Tf yes y ask the following: Vlzojbz dtsc/Ubz thz^natun.^ 

tht addLtiondl loo^xk; iut,, 'individual ttUoHA^ng, 6maiZ ghoup- 

i^)on,k, math lab, oXc, i 



-4r- 



. Question 12: The study we are planning will deal only with chiWren al^he 
fourth gr^de level . >Jhe rest of the questioTis^^^lJ^at^^ 
' ^ to do with your program for Tourth grade level "cFtfldren. 

Me youA ^oiinXh gnad^ c£a64e6 gn^odptd hy abitcty on, oAe thzy 

, I by ability 

j heterogeneous 



Question 13: thz math ctin/UcuZm u^td in youJi ^ouAth g^ada IzveZ ^pzcl^izd 
in ttm6 'o{^ btha\)ion,ciJi objzctiveA? " ^ . - 

Yes ' . - 



No ■ 



Question 14: U tjiz n.zadLng cuMXcciLtu/n ^pzCyL^itd in tznm hthavloKol 

Yes ■ . " 

No ' : -1 ' 
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' Schoal I ntfrrview Form- Individualized Site (5) • . 

Qu€St4on 15: Do thWiKzn gzneAotty bag-in wonk out ik& bzglnyUng th& ickool 
yzoA at dl^zAtnt platti in tke. math. auoUculm? 
Yes ' 

No ' ' ■ ' 



Question 16: Jnjuading? 

Yes 



' L. 



No 



.Question 17: 0oe4 th& math pjiogtam make. f^ov-Uioru ^OK ikdwidtLoZ gating''. 



□ 



No 



Question 18: OoeA youA ' fieading pxogkam? 

□ Yes 



\ 



Question 19: (a) ckiidAeA- gtmiAolly bzgiyi a new math topic at tht J^amz 

.time,? . . , . . * 
.Yes 

No ^ - ' ' ■ - • . . ' ■ 



□ 
□ 



(b) Hoijo do tzachvu dtdiAt that thz paplts oaq, KO^ady to bo^gin 
a MM) innath topic? ^ : 



Question 20: (a) Vo childKoyi QQjnViaZti) hzqin nm matVbial^ in Ke/iding at tht 
' -dome tmz? • ' * , ^ , ■ 

Yes, • ^ » ■ , . 

No • ■ • : ■ "v 



□ 
□ 
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' 'School .Interview Fprjn-IntiiWidu^ . 

• ' Qljestlon. 20:^ {b\ ' Hotjo do^ tzackoM^. ddcLdzJtha;^ tht papi^i-.oAe ^^dy to go on 
' *^ >, to new matojuat? ; . . 



* / Thaijlc- ybu for your help and responding to- this questionnaire. After we 

haye reviewed .tbeL.resuVts of our initial screenrng- process , we will be -in ' 

tptich. with' - r • / with regard to the, selection of possible 

,] ] /A^-^ district office contact" * ' , " 

'Study.''^ites.^/We hope that you^ would b^ interested in participating in the 

^ : ;^ . - f ' ' ' ■ " ' * ' ' \ ' ' ' 

study. ^ Agarn», tfenk you' for your time. ' 
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Classroom Environment-* Screening (CE-S) 

R, S. Soar 
R. M. Soar 



1. What proportion of the affect expressed by pupils in the 
classroom is negative? 



almost none about 1/4 about 1/2 about 3/4 almost all 

L L : / . / L 



2. What fraction of the time do pupils make it necessary for the 
teacher to resort to conmands and criticism to maintain order? 



almost none about 1/4 about 1/2* s about 3/4 almost all 

L L \. / / , L 



Z., How much of the school day does the typcial pupil speVid on 
structured learning tasks, either alone or in a group? 



almost none about 1/4 about 1/2' about 3/4 almost all 

L L / ' ' / _L 



A. -How much, of the school day does, the typical pupil spend 6n 
tasks assigned by the teachfer, in contrast tb tasks in which 
he has some choice? 



almost none about 1/4 about 1/2 ' about 3/4 almost^all 



5. How much of the school day does the typical pupil spend working 
alone? . . . t ' 

almost none about T/4 - 'about 1/2 about 3/4 'almost all 

L / ' / ' : / 
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Cl^sroom Environment— Screening (2) • * . 

6. How much of the teacfter's classroom management conveys 
accepting, val uing. and respecting • Individual pupils? 

almost none about 1/4 : ' about 1/2 about 3/4 almost all 
L , / , / . / / ■ 

*' " ' ■ 

7/ In what proportion of activities is the typical pupil 
permitted to whisper or talk to other pupils? 

almost. none ' ■ about 1/4 - about 1/2° ' about 3/4 '''almost all 
- L -J^ I \ ■ / • / . 



8,. -After finishing one activity, how many minutes does the 
typical pupil have to wait for the next activity to begin? 



1 or less '3 - . 6 9 12 or more 
1 : L : /• ■ / L 



9. On the averageji^ what proportion of pupils seem interested 
or involved in 1;he ongoing activity? 



almost none about 1/4 about 1/2 akout 3/4 almost all 
L L . L L :± 



Rationale 
for 

Suggested Screening Vartables 



The rationale on which thef screening ratings are based 

I' ' " 

Is "bKo-fold." ^ ^ \ 

First, they are ratings for w^ich low inference measures 
already exist in present observation instruments, with Varying 
degrees of parallel for all but on^ (the eighth). This choice 
was made because of the wis.h to be' able to relate these rating 
measures from District Survey I- to observation measures which 
would be obtained as part of District Survey II. The decision 
to avoid high inference observation measures Is based.on the 
difficulty t)f knowing what- behaviors they represent. For 
example, Rosenshine and Furst (1973) identify Clarity as an 
Important variable in teacher .effectiveness," but then conment,. 
"Unfortunately, it 1s not clear just what is m.eant l)y clarity, 
and future investigators niight well attempkt to determine the 
more specific behaviors which comprise a high rat1ng.,on clarity, 
(p. 156). While it is true that an individual researcher might 
train observers to criterion reliability with high Inference 
measures such as Clarity or Enthusiasm, this 1 eaves .untouched > 
the difficulty of whether variables with the same names used by. 
other researchers would measure the same behaviors. In addition 
the problems of implementing such variables in training program^ 
or by individual teachers in improving their own teaching are 
formidable. For these reasons,' the proposed screening variables 
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ar^e ones which we believe un be tied to low Inference measures 
In existing observation Instrurients. 

Beyond this, first of all, ratings were selected' for 
which there-is some empirical evidence of relation with' pupil 
outcomes, or secondarily which represent current beliefs about 
effective teaching. For exampTe, the ratings which reflect 
the time the pupil spends on structured learning tasks seem 
related to the variable Student -Opportuni ty To Learn Criterion 
Material which Rosenshine and Furst identify as important, 
with additional" support from Soar (1973) and Stallings (1973). 

. The measures which reflect the emotional climate of the 
classroom in terns of both teacher and pupil behavior seem 
related to the variable Criticism icited as important byRosenshijie 
and- Furst. In'a'ddition, work by Soar and Soar (1973) and 
Brophy and Evertson (1974) agree in Indicating that a positive 
emotional climate is more important .to the achievement gain^of 
^disadvantaged pupils'than to middle class pupils. Further,- there 
is strong agreement in current beliefs about best classroom 
practice on" the desirability of a positive emotional climate. 

- The rating of proportion of activities which are assigned 
by the teachfer in contrast to those in which the pupil has choice 
is based on work (Soar^and Soar, 1972, 1973) showing a relationship 
between teacher dirlcted act1v1t"1e$ and pupil achievement. gain. 
This relationship! was in the form of an inverted "U", in which 
either higher or lower amounts of teacher direction were associated 
with less pupil gain than intermediate amounts. This would 
suggest screening; out teachers at either extreme, but we ..would, 

0 V 
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recownend eliminating only those who assign or direcMlttle 
of the pupil activities, on the assiiflptlon^that their pupils- 
ma^y have relatively little expbsure to learning ^tasks.. The 
ftnding that higher amounts^ of teacher direction were associated 
with decreased learning my have1)een caused by the failure ' 
of the teacher tp assign tasks that appropriately ipet the pupil'; 
need for learning, while a degtte of choice permitted the -pup^l- 
to, fit the task to his needs. This difficulty might be remedied 
by a well^implemented individualized p.rograni. ^ 

The ratings whkh identify amount of pupil socialization 
and the extent to which pupils work alone have empiriCaV bases 
which are. only suggestive but they reflect current beliefs 
about educational settings which are conducive to pupil social 
development., Low inference measures of both of these exist* - ^ 

The rating of interest and involvement of pupils is based 
on data indicating that interest is related with gain in achieve 
m^nt, and with other process variables, such»-as the emotional 
climate of the classroom, which are valued in themselves (Soar 
an^l So?[r,.1973; Brophy and Evertson, 1974), 

Cut-Off Points - ' ' ;\ 

If cut-off joints are needed we wo^uld suggest eliminaiting/ 
classes rated as>"almost none" for ratings on items 3, 4, 6 
and 7; those rated "about 1/2" on more for item 2; tfiose rated 
"almost all" for item 5; those rated "9 or more for item S-j 
"3/4 or mprev" for item 1 and "about 1/4". or less for item 9., 
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TEACHER'S QUESTIONNAIRE (Screening) (TQ-S) 

Teacher's Name ' 

.School ^° •- > - • • Grade' Level . 

District 1 \ : ' ' -Ungraded. Ages ^ 

State - ' " * . • 

Content Area (Math or Reading) ... 

.* f 

Me are interested in, determining some of the ways" in which- 
you attend to various instructional tasks that 'we believe are ' 
coirtnon to most instructional contexts. First we will ask softie 
genera> questions about curricul'um for Math/Reading. For each', 
instructional task, we^ would like to know the size of the group 
affected by sopie of your instructional decisions. 

• v. 

1. First with regard to curriculum scope. Would you say/that 
your students -would work in graded curriculum, that fs: • 

a. Appropriate for the grade level only.^ / 

b. Appropriate for two %r more grades, but^ess than 

, , the entire 1-6 curriculum. 

c.^ Equivalent to an entire 1-6 grade curriculum. 

2. Would you say that what you intend to teach this year will 
• be based on expectancies for: - . ~~ 

< » * 

; a : The grade group 

b. Your class ^roup as a whole. 

c. Subgroups of your class. • 

d. ' Each learner independently/ 
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Teacher's Questionnaire (2) 



3. For your class, wtll the sequence of learning tasks, most of 
■ the time, be: 



a. Essentially the same as for other class groups of 
the same grade level in the school . . 

b. Essentially the sam"§ for your "entire class. 
c\ Different for subgroups of your class. 

d. Different for individuals. 



4.- Was your decision about where students were to start in 
. the currfculum: 



a. Essentially the same for your class as for other 
- classes in the same grade in the school. 

b. The same for al.l m&nbers of^your class.- . 

' c, Bifferent for different subgroups of .your class. 

d, Diffjerent for each Individual. ' 

5. -When you de'cide^what t5 teach next during the year, will 
the decision, most ofteli, be made:. 



a. For .your class as. a whole.- . ■■• ' 
t. • • • ^ 

b. For. different subgroups in your class. 

c. For each individual independently. 

6, In evaluating learning would you say your decision most of 
the time; depends on the performance- oT4 



.a. The-typlcal grade lev6l group = 

♦ •• ■ ■ , , ' ■ 
.b.- .Your, class group as a whole. 

c. A subgroup of your class. 

d. Each child individually. ;'- 
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Teacher's Questionnaire (3) 



7. In your classroom when you af*e evaluating learning by 

ackninistering a test» would it be true most of the time that: 



a. Each learner would be taking a different test. 

fe. A few groups smaller than the class would be 

. taking different tests. ' ' 

c. The wh^le class would be taking the sanie test. . 

8', Would you say that the time you can make available for the 
pupil in any given instructional' segment is dependent upon: 



' a. Grade content to be covered. 



b. The time the jnajority or substantial portion of 
the class heeds to learn the lesson. 

The time the majority of a class subgroup needs > 
.to learn the lesson. 

d. The time each Individual needs to learp the lejsson. 
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INSTRUMENTATIOK (continued) ^ 
2> Observ^ation Forms 

- ' • Instructional Task Treatment Observation Form' 
(ITTOF)', . ' 

' -(a) Summa)*y of the Pilot Test 

' '(b) Suggested Training Program' . •' 

• Florida Climate "and Control System (FLACCSX 
- . • .Classroom Global Rating (GR) 

. ^ • Teacher Practices Observation Record (TPOR) 

• Classroom Description (CD) 
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INSTRUCTIONAL TA^-I^EATMENT 
OBSERVATIONAL FORM .(IHOF) • 

(Trial Version - TO/17775) 



Name of Observer _____ 
Date of Observation ." 
Obser vat i on/ CI as s . Number 
Names of Other Observers 



SUMMARY 

1.0 
2.0 
3.0 
3.1 
^ 3.2 
4.0 
5.0 
6.0 
7.0 



ERIC 



8.0 

8.1 ■ . 

8.2 

9.0 

9.1 ^ 

9.2 

10. 0 ^ 

.10.1 

10.2 . ■ 
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1.0 PROVISION OF CURRICULUM OPPORTUNITY 



Instructional Unit Size 



• Level of Attention 



The teacher 4u:9vides--a curriculum The teacher provides a curriculum 
opportunity: opportunity for: 



equival.ent to 3 -or more 
grade levels - 

equivalent to two grade • 
levels 

equivalent to one grade level 



3 each individual in the plass 3 

2 each subgroup in the class - 2 
1 the class group as a whole 1 



^Scope of: 



For: 



Graded content 



Three or more grades 
Two grades 
One grade 



Individual 
Learners 



9 
7 
5 



Subgroups 



8- 
6 
4 



Class 
Groups 



. 3 
2 
1 
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2.0 -STAT^ENTS OF CURRICULUM INTENTIONS 



Instructional Unit Size 

The curriculum intentions are 
stated as: 



Level of Attmt1#n 

.The curriculum intentions 
are stated fof: 



pupil content behaviors 
content offerings 

broad goal statements 



3 
2 



individual learners 

class subgroups j 

the class group as 
a whole 



3 
2 



Specificity of 
Intentions^ - 



For: ' 

Individual 
Learners 



Subgroups 



Class 
Gif'oups 



Pupil Content Behaviors 
Content Offerings- 
Broad doal Statements - 



5 . 



8 . 

6 

4 



3 
2 
1 
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mOF (4) 

3.0' CURRICULUM PLACEMENT DECISION 



■Instructional Unit Sizg : 

The. teacher makes the placement 
decision for: 



Level of AtterTtion 



The teacher rn^kes the 
placanent dej:'ision by] 



learners independently * 3 



subgroups separately 2 

for the class group 

as a~ whole 1 



criterion' referenced . 

performance ^ 3 

judgment 2 

grade level 

exinctatlons 1 



Decision Based on. 



For: 



Individual 
Lei^rners 



Subgroups 



Class 
Groups 



Cr i teri on-Ref erenciid 
Performances 



Judpent 

Grade Level - 
Expectations ^/ 



8 
6 

4' 



. 3 
2 

^ 1 
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4.0 ADJUSTING RATE OF INSTRUCTION . 



Instructional Unit Size 
Teacher adjusts the rate of Instruction for: 



individuals 
subgroups 
class groups 



3 
2 
1 



Individual 
'Learners 



Subgroups 



Class 
Groups 



4.0 Rate of Instruction- 



f 
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INSTRUCTIONAL ^^XPERIENCE- 



Frequency of Attention 



High , 

□ 
□ 
□ 



Moderate 



Low 

□ 
□ 
□ 



5.0 Provision for 
Individual 
Responding 

6.0 Provision for 
Individual 
feedback 

7.0 Monitoring 
Individual 
Progress 
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8 . 0 PERFORMANCE STANDARDS - FOR ADVANCEMENT , - 



Instructional Unit Size 



\ ' Level of Attention 



8.1 The teacher applies stanTdards 8.2 The teacher applies: 

for-.ailyance6ent..±a:-^. 1^, J.^-^ 



•individual learners 3 
subgroups of the class 2 



the class group as a 
•whole y 



an absolute - 
performance -standard 3 

^a variable 

performance, standard ' 2 

ho performance ' - 
standard 1 



Performance 
Standard is 


Applied to; 

Individual 
Learners 


Subgroups 


c 

Class . 
Groups 


Absolute 


9 ' 




3 ■ ^ 


Variable 


* 


' -. 6 




Not Established- 


. 5- 


' 4 


. r 
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9.0 EVALUATION OF PERFORMANCE- 



Inst* 



)nal Unit Size 



9.1 The teacher, evaluates ' 
.performance in terms of:/'. 



Level of Attention 



9.2 Th6 teacher gValuates 
performance in terns, of 



, individual learners 


3 


program prepared test 


suharf)un<; ''a^' a wHoIp 


? ^ 




tne Class groups aS 
a whole 


1 

» 


^judgment 


% 

Evaluation Based On 


.For: 






Individual 
Learners 


Class 

Subgroups Groups 


Program Prepared Tests 


9 


' 8 ■ 3 


Teacher Prepared Tests 


7 


6 2 


Judgment 


5 


. 4 1 



3 
2 

-1 
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4 



4 

<» 








% < 
* 

* 


• ITTOF (9) 


* 






10.0 ^MATCHING LEARNERS WITH NEXT 


INSTRUCTION 






.Instructional Urtit Size 


Level of Attention' 


• 


• 

t » o 


. 10.1 The teacher matches 


'10.2 The teacher^ matches 






ledrners with instruc- 


learners with instruc- ' 






tion for: <^ 


tion hyi 




i 


individuals separately 3 


* criterion-referenced 








performance 3 






subgroups of the class 2 


judgment ' 2 *^ ■ 






the -class as a whole 1 


position on standard 








' curriculum sequence 1 






Match with Instruction 








Based On: 


For: - ' 






* * 


Individual Subgroups Class 








Learner Groups 




• 


Cri ter i on-Referenced 








Performances 


9 8 . 3 - 






Judament 


7 .6 2 / 


• 




R9^1on in Standard 








Curriculum Sequence 


4. ; " 1 
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• a.- SUW4ARY OF PILOT TEST OF THE INSTRUCTIONAL TASK 
TREATMENT OBSERVATION FORM (ITTOF) 

. Once it became clear during the planning of the study 
thjtt it would be necessary ta develop a new instrument to 
gather data regarding ^the pi^ogram variables, plans were made 
to pilot^ test the ^sjrjt^t^ion^al^ Tas^k Treatment Observation _ 
Form ClTTOF) in various classroom, settings. It was>ecogni2ed 
that, due to limited time and resources, a full and adequate 
pilot; test of the ITTOF would not be possible. It was felt 
that such an effort wpuld be useful in order to refine, as • 
mch as possible, the instrument itself, the suggested, 
training' program, and the recommended observation procedures. 
Because of the limited time available for training, it was 
decided to utili2e highly experienced curriculum developent 
experts who, by reason of 'their ^tensive experience and 
familiarity with developing, testing and monitoring of \/ 
individualized programs, would require a minimum of tra.intng 
tifne. It was also reasoned that this type of observer, while 
far moVe experienced than^ might' be required fcfr the actual 
study, ^ould more likely be able to make positive contri- . 
butions to the improvemen^t'of^the instrument, the procedures,* 
5ind the training, program. 

The objectives of the initial pilot test of the ITTOF 
were as follows: 
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(1) DCTonstrate that moderately-high 
inference observations of program 
variables could be made in a variety 
, of classroom settings, 

*^ '(2); Obtain a. critical reV'iew of the 

• ^ / procedures for' administering the ITTOF, 

(3) Obtain 'in^ut for the improvement of 
the training program for the ITTOF, 

\ . V^ .Obtain ^ jpeasure of inter-rater agreement ' 

^ J for xRe ^liuN^aM ' ' ^ 

♦C5) :0bt?iin some preliminary general izability 
' • data. 

' Six^ejcperienqed cUrriciilum development staff were selfected 
. by the study designers on the basis^of their classrbom experience 
Wi-th the- developrpent and* testing of individualized instruction 
in the basic skills. - An ini^fa*! training session of three hours 
was held to provide the necessary background for the. study and / 
the ITTOf . . ^ * ^ 

During the initial training session , the primary investi- 
gators explained the instrujnent ancl its rationale and "defined 
terms, following ^the general introduction, to* the ITTpF>,. there . 
was extensive interjaction with the observation te^m and the" 
instrument designers. T^e bulk^f, the sj^ssion was recorded for * 
use in restructuring the training program at a later date^ 

Following the primary training session a first round of. ^ ^ 
observations was conducted in' an IGE sjetting known to^be well-" 
"implemented. Each observation session consisted of having - ■ 
three observers operate independently in thfi same classroom 
during the sanie lesson." Following the lesson, each observer 



would speak private!/ with the teacher to complete- the 
observation. During the a fternoo'n following these visitations 
a second training or debfiefing session was held in which 
observers and the ITTOF designers were able again to interact 
extensively. A second and a'\ third s.ession of this type was 

held following success ive observations in a variety of math 

and reading settings.^ What resulted from these sessions- 
contributed substantially to the sharpening of definitiorrs 
and improvement of operational procedures.^ - As these 
experience? continued, there was a noticeable increase in 
inter-rater agreement for the same classroom period. 

For those sets of observers who (1) made' at least one 
prior observation using the ITTOF, and (2) attended a- 
debriefing session following such observations, the inter- 
rater agreement ranged 'from a low' of 64^ to "a high of 100^. 
'And, these agreement percentages were for IPI (93%, 100%), 
IGE (79%), and standardized (64%, 93%) classes in both math 
and reading. In one instance inter-rater agreement was only 
22% and it was determined that the problem arose- because of 
a severe disparity between what the observers thought they 
observed (50% agreement) and what the teacher said was going 
on during the class session. Observers in this case, tended 
to resolve the disparity in different ways. While this 
remains as a possible problem, we are c9nfident it is 

2f)4 ^- 
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resolvable by jDreparing a pool of questions that might be. 
asked of the teacher, . ^ ^ 

All but one of the twelve classes observed during the 
pilot test'Were fourth grade reading or mathematics. The 
exception was a combined multi-age group for grades '5 and 6. 

&yr4j>§-4te~^4^ts^-iper4^v^bserver-s-o^^ 

watched lessons, examined materials, studied student work in' 
progress spoke with students, and observed teacher-pupil, 
pupil -aide, and pupil -pupil interactions. During this 
period, observers made notes regarding the program variables. 
Following the class period the teacher in charge answered 
questions raised by the observers. Since the questioning^, 
of the teacher does not follow a specific interview schedule, 
but rather is based upon the observer's perceptions made 
during the class period, these interviews were done independently. 
The interviews lasted approximately ten minutes each thus 
requiring about 30 minutes of the teacher's time away frbm the 
class. For this reason, in most cases, it was necessary to 
arrange for classroom coverage in order tq have adequate time 
with the teacher. In some instances, it was necessary to 
arrange for payment of the teacher for the observation/ interview 
process, particularly where arrangement for coverage was a 
problem. ^ 

The ITTOF form was filled out by each observer independently 
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following the ;:onipletion of the observation and the teacher 
interview. The names of the' observers' and the classes 
observed were identified by a numeric cpde to insure the 
confidential nature and objectivity of'the data. 

The specific recommendations of the observation tern 

for the pilot test were: \ 

c-. — . ^ — . — ^ 

(1) The training program should enable observers 
to obtain some preliminary ejcperiences in^a 

\Variety of Instructional settings and a de- 
briefing session following each such 
\ observation is recommended to achieve agree- 

ment on subsequent observations. 

(2) Prior to' making observations a brief overview 
of the program would be helpful. 

(3) Also prior .to making observationsS observers ' ^ 
should be familiar with the materials, objectives, 

, test materials, etc. 

(4) A poftl of possible questions to be used with I 
when interviewing teachers should be helpful. 

(5) A pool of possible questions to be used with 
students should be helpful. 

(6) A clear set of definitions should be developed 
to help observers make judgments. 

(7) Rules for arriving at a consensual agreement 
when observational and interview data are in 
Conflict is needed .< 

' (8) A set of mini, case studies possibly with~ short 
video ta^ed classroom sequences would be helpful 
to observers during training. 

(^9) Several video tapes of entire classroom sessions 
and teacher interviews should be prepared for 
training. These, would use a "through the eyes 
of the observer" approach. 

In summary, the pilot test of the ITTOF achieved all of the ' 

intended objectives. There is the clear indication "that it will 
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be possible to make these moderately high inference 
observations with a fairly high degree of inter-rater 
agreement^ It appears that observers who were able to 
participate in several- Successive observations and de- 
brief ings-were able to achieve a high level of agreements 
JUnfortunately, because of scheduling problems and, 
availability of pefrsonnel , it was no-t always possible for 
an observer to attend complete, debriefing sessions prior to 
the^next observation.) It is felt that this will not be a • 
problem to the study contractor since there should not be 
conflictihg requirements' on observation personnerduring 
the s-tudy. 

It is recommended that during the Spring of 1976, the 
contractor undertake a more intensive field testing of the- 
ITTOF for two purposes. Firstly, to train the personnel 
who will be training the actual study observers and, 
secondly, to establish acceptable general izabiTity and 
inter-rater agreement figures. Such an effort should be 
undertaken by a group of curriculum, development experts 
having extensive classroom observation experience, 
b.^ SUGGESTED TRAINING PROGRAM FOR THE ITTOF ' ' 
The training program for the use of the ITTOF should be 
developed by the contractor during the Spring of 1976. the 
devel6pment should be done by the same team of experienced 

■ ' . ' . ■> 
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curriculum development personnel who will participate as 
classroom observers in the field test of the ITTOFF durirtg 
the early Spring. 

• The training of observers will involve three major ^ 
phases. The first or introductory stage will introduce 

as it has been defined for the study. Since it is 
recommended that the observers have prior experience as 
classroom teachers, the instruction will contrast various 
forms of individualized Instruction with standardized 
instruction. Where possible, teacher training films will 
be used to provide a commonality of experience among the 
observen^.._ The^introduction to individualized 1n§truction 
will involve work in the training programs for the njajor 
individualized programs, IPI PLAN*^and IGE. Representatives 
of these programs will make presentations to the observer 
group and again training films will be used. At the 
conclusion of this first phtfse of training, each observer 

» 

Will have a working knowledge of the major individualized . 
programs and will be able to contrast these with standardized 
programs 'in terms of the ten major program variables. 

The second phase of the training will involve' the 
introduction to various classroom observation techniques, 
followed by specific 'Work on the use of the ITTOP. Training 
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will be done with a series of video tapes of act)ial 
classroom situations. These tapes will be prepared during 
the Spring of 1976 using -the "through the eyes of the 
observer" technique which will focus on those things an . 
observer would be expected. to focus on during a class 
observ a tioo. — lape-s-viUJ-aJ^so -ijic-l-ude-iRJterviews with 



teachers. Observers will use the tapes to complete an 
ITTOF rating which will he discussed with the other, 
observers and the instructor. During the training session 
video tapes- will be viewed in order of Increasing diffi- 
culty with the first tape showing clearly identifiable 
Individualized, or standardized oehavlor followed by tapes 
-that show situations more difficult to judge. At the 
completion of phase 2, the observers should finally under- 
stand the ITTOF and be able to attain high inter-rater 
agreement from video taped classes. 

The final phase of training will occur during the last 
two ^ weeks in September and the first week in October of 1976 
when observers will b'e broken into teams of six and will 
observe classes in much the- same manner as was used during 
the in.itial pilot test during the planning contract period. 
Each class observation and teacher interviews will b^ clone 
by the observers and will be followed by a debriefing. The 
observations will not occur in study classes since during 
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this same interval pretesting will be occurring in those 
classes. The purpose of this final training stage will 
be to give observers actual classroom experience in the 
use of the ITTOF and to demonstrate high agreement among 
observer ratings. 
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'institute for the development of human resources 

University, of Floridn 
Gaiaesville, Florida 



FLACCS* 

Florida Climate and Control System 



Program 
Ci tv " 



School 
Grade 



Teacher 
Date 



.♦Observer 
Series 



Abundant L 
\ Varied 



Children'* s Art Work Displayed 

Quite a few , Some , ' A few 

4 ' ' a . 2 • 



None , 
1' 



Most are 



if Relation of Room Displays and Artifacts 
* To Children's Subbultaral Background 



clearly 


Quite a few ' • 


Some a tx> " 


^ A few are ' 


'None -are 


.Not ' 


related 


are^ related ' ' 


related 


related 


related 




5 


^ 4 • • ' . 


3 • 


2 * -V 


/ 1 - 


To 




■ - ■ ' , ■ ■ , 

• .^.-^ 











♦This is ?in expe^j'imental form which should^not cited %' 
' ,3 or.uAcd wfthout permisVj.oa.ot the dovolopors. * '\\' 
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^ Florida Climate and Control System - Cont. ^ 

— r 1-3 Deck Nuiohor G 4-7 T. N^mbct* ^ C R Grodo C U flcl ^nbi 1 i ty 



/ ^ NEGATIVE AFFECT Series Tenchcr 

I 

Verbal \ ^ Toother Nonvcrhnl ^ 





I 




3 


K C. 


Toq 1 


2 


I :j 












Savs "stop U,\ f?tc, 10 










WflAtn idr chi Id 










L'bc\s throa tcin ni^ tone 1 1 










Frowns 










){i-ierts child 12 










l\M nis, shakes 1 1 n;:er 


t.it 








Ci" I L » r ; 7.0S . bl aines I 3 










P»ir»hf*s or pulls, hohi.^ ^ 










W.irns 14 










:Sho'.Ts (1 ) schist 


li 








Ye lis 15 










Takes na te ria I 


ri.i 








Sco] ds , huinl 1 1 a t cs 16 










Refuses to respond to chi Id 


1^ • 




> 




.Other 17 










Other 












Co d 1^ Involvement' 






Verbal Pupj 


1 Nonverbal 


P' 




j 1 jSn\s 'No/' "I won't" etc* 






1 jMAUes laco, 1 rowns ^ 


?0 




1 j 1 Ten SOS 19 






1 1 Pours, v/i T hdraus 






1 1 ) I*Hi|:hs , « 20 






1 j I'Dfoftpora I 1 VP , resistant 


a 


"j t 1 rnM los 21; 




j jStanps, ihrnu's, .slans ' 


:>:ii 


\ 1 ICor.-nnnfis or d*>nnnds 22 




j 1 I n c e r r o re N . i h rea*te ws - 


'iili 






or i 


1 -n^^nnnHQ '>»t:r>ntion 24* 


1 1 fPirks .at c^ild 




i 


IMakes someone 'feel small Zt>* 


I 1 1 hMsnes or pn 1 1 s, nohis 


27! 




jFjnrlR fa\Mt ' 26- 


1 [■ 1 Hi t:». hurt a. 




1 


|Tlvreatens o 27i 


1 1 ^1 Is loft out 




, 1 


1 Other. /• - . . 2H. " 


1 1 i^thor 




^ 1 


ICade Involvement ' , - \ « 


• 


i positive: AFFECT . ' . 
v' • ycrbal ' \ ' 'Tcachtr , Nonvorbpl 


31 




^Sc-Tvft,/'TbnnK \ou/^tc, 29j 


-f lAcceptfi' favors for self 


A?. 




j j iAfrfecs with* chi l(i , 30j 




\^ 


I^Aits 'Tor ch]lri 




j 1 |Gilr5po;-Cs chul^d- , ^ 31^} ... 






•|fii vea .1 ndi v idun 1 a t tent ion 




j. 1 '|Givds -inclfv^dual atte'atjon^ '32i - 


¥ 




'* !Wam,, concronial - 






11 ,|Warn, 'comx.oni-a 1 * • , 33j 


9 






Listprts carefu Uv lo child 






, ^ :| IPraKSes^ chi Ul • 34t. 








Smjiea, lauchS, nods 


37 














Pfit5. hufrs, etc. • » 


38 








is' enUmsiast.ic 36j 










S-j-npatheti c- 


30 




r, 1 1 1 


0chi?r' , >^ 37 










Othcfr . / ■ 


40 






Code. I n vol V oijie n t ' - ^ ' >v 


7-T r / 




* • Verb'al- 1 Pu^i^ 


U . , ' Nonverbal 










^JSuvs "-Thftnk vpir.l etc.- 38; 






iHe^ipfu 1 , sliaro.s 










|.Soun(f'j^ iricndly ^ .39jll 






- |4x?nns ; e lo*;o to another 


'1-3 








• jAnVeps th anot^ier »^ 40j ' 






^|Chofysei^ another ' . 


4. 1 








^ fn I J a to$ rorvt^act ' / 4 1 j 






j\snllei?, laiicrbs with another ' 










s^'Of fers ,to, ^horc% cooperate. 42| 






• j Pa t3 . .'J'tifiJ^^ ano't^ier 


•16 








'*j^?uppdr ts, pnothcr 43 








JAKreeatkle, co(^pera 1 1 vi? 


4/ 




t 




*| lU ,0 n t hn s rn^s t ) c , ' ^ 








* J/nthtis 1 as ti r 








1 jf^rai fic.s -arro ther . 15 








jHottseplaV ' yT- 








•j jjllOlps ^ooLlicr . ^ , 46 








[Other 








^ ^|(itht?,r - ^ . ' 47 








• j ('Co-htinu^d over)_ 




V. " 






»» 



U ' < " • ..... •' o.p.r;, . <»DE lKVOLiraEr*T 

^ V;-- *' - . 1%^ - ' . * ^ 0. None 1nvolvo(f 

\\'^'^'t'/'\ . . ' ''V • ' ^ f^^^ involved * 

• > ^ . * • 'V-50 ; 2u Up , to \ the clabs 

- • . 7 .\/ ' ' .KUne. than ha> f . % 



C 1-3 DccU Number C 4-7 T. Nmnbcr \ t 8 Gr:ulc C V lu* \ ^oh) I i i y or 1) 

(2^o\n. 1st) 



C. I Tot l\ 2 [i THAcrtEK 


c. 


TirL- 1. 2 ;{ 


PUPIL 


10« j 1 ' r<*nchcr Ccn\ rn 1 


10 




Pupi 1 Ci'Utra 1 


111 1 !l/^n(].s hi ni' 1 nfx , ^nmos . storvtm 


11 


1 ! 


Pup I I — no cl^oice 


)2| 


• « *U)-, cs ) rcp 1 V Qmoiirr piM) ) 3 s 


12 


• ! 1 . 


Pup\l 1 mi led (linice 


13| 


1 ! V/ \ t hcl rnw s f i-( >ni c I n s s 


V^ 


; 1 k. 


I^)p I I 1 ree c^-.o i c6 


l.J| 


'Uses hltTcl;l)onrcl, A-V Kqujp, 






151 


' I frnorcs, rejusos to att-enci P. 


M 


' 1 


(*Seat work w/o teacher 


Ui| 


1 lAl tencJs P, ^^br leflv 


15 


! i 


(*S(»at work wj t!h teacher 


17| 


\M lends ' P. closolv 






18| 


•AitencJs P. 2i> buccession * 


16 






(♦Works » plays w, much supn^ . 


19 




1 • 'Attends siinul tnneous acliv. 


17 




i 


(♦Works', plays w, little supv. 










Ift 


! i ! 


(*li(>sisls, disolieyb directfbns' 




1,1' Pr.li ,ses 


IJ) 


i 1 • 


(^Oiievs 01 rect 1 ons 


21 


; • « for smtus 


20 


1 1 ; 


Asks pormssion 




1 , .Mi'.uosis , c!cs 


'>A 


1 • 1 ! : 


Fnllnv.s rnutin^"* w/o remindc^r 


23 


1 ' . r'ocflhHv^k , c) les' i'o:^son 


22 


! 1 . 


ivcporis rule to anott^er 


2-1 1 1 . L)»j'.>Gt ><)iis Jor re f 1 11 vc», ihot 


23 


1 , 1 i'mitics 


::3 ' (■ vr- ' cr'tM'i^T (,'.V;) '.'i: ' . f*^ in'op'Otion 
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: Questions for control |25| 


.j 1 f Gi ves di rec 1 1 on 






>ouesiioas oca'tes ucn.ruie (*^o 


1 ' 


j ' j o J V L. j> 1 ed:>un 




• !* Uirccis wiiji roa:>on n2l7 


• 




j opeJ'kii 11 1 (Hid w/o j)errnissjon 






: 1 .Directs w/o. reason {28 


■ 




1 Kne:nces in ou t-of- tx-junds beh. 


JO 




« 


U.ses tine pressure |20 






j Col 1 al)orn ICS w. ien<'her 


31 






Cnll child.bv none (KV.'S) ^30 




1 1 Task i-elatcci rno>-enent 


32 




! 


Interrupts Pupil. cues oil 


31 






J /^unless uantierin^' 


33 




1 


v;.)rns 


32 






t Fantasy 


34 




1 


\ Supv. p". close ly . imbl) /-OS. 


33 


1 1 




j Uses pla\ object as itsell 


35 




1 


' Cr I ti ci7.cs 


3A 


1 


u 


1 Parallel plnv Dr work 


36 






' Orders, commrtnds 


35 






] Works, p lavs co 1 la l)ora 1 1 ve I v 


37 






"^Scolds, punishes 


36 






j V/ork^, nlays ronpe 1 1 I M'e I v 


3B 




1 


i)hps Xirm tone 


37 






j Seeks reassurance, support 


39 




1 ; Uses bhnrp lonn , 


3ii 






j Shows pride 










39 








Shows fear, shame, hvirni 1 1 a t ion 










40 






j Shows apathy 




NONVERB/XL CONTROL 




• WORK GROUPS 


40 






To*le)*otes tioviant bch. 


4L 




• i 


Pupi 1 as md) vidua I 


41 






Positive rection 


12 






Group w. teacher 


42 






{ No<I«>.^ smiles for ctkntrol 


43 






Structured c^roups w/o t . 


43 






1 Positive farm I feedbnck 


44 






Free .pcronps 


4 4 






J V.^os body Knr:! 1 sh ' 




> 

' SOCIALIZATION . , 


45 






j G(>s cures 


46 






1 Gi^es tanpoble reward 


47 






j Touches, pots (jreille) 


145 


I 


j Almost* never _ 


48 






1 Holds , pushes . spn nks (hnr sh ) 


46 


1 


j Occa SI ona 1 1 v 


4f) 






1 • TnUes 'pric^nL, book 


47 




1 Frequ(^nt I v 


50 






1 b I t:nn I s , rn))S 




'^^m)?^riai5 


51 






1 S)ihh.' Shnl:(^s bond 


52 






1 , G 1 nr(»s , f rowns 






u 


i 


Striu'ture T. hf.4iav)or 


'I'f; 


1 . 


1 


Struyluro P, bel^avior 








PUPIL INTFI!'^ST ATTKWTION 
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Classrogm Global Ratings 



4* Pupil Groupings ' . 

. , ' Emerge about half More often 
Fixed .and regular Mostly^ the time, fixed emerge spon- Usually emerge 
for activities j'iked half the time taneously spontaneously 

/ 4 5 



Almost always 
vork at same 
activity . 



No feelings 
expressed , 



Unaware of 
jconspicxious 
i < feeling 

- 1 



^ 2 . 3 

Pupil Differentiation. . 

>tost work at - ^ost work,a,t 

same activity' same activity 

most of the ' half of the ^ 

time ' time - 



Worrk at different 
activities more 
often than not ' 



2f . ^ . S 
- Teacher^ Congruence 



Words clearly contra- 
dict evident 
feelings . 



Some agreement 
of words and 
feelings-'' 



1 . .2 3 4 

Teacher Empathy 

Occasionally . Sometimes aware 

aware of ob- Usually aware of . of subtle ^ 

vious feeling obvious feeling ' feeling 



Usually work 
at different 
activities 



Words and c 
feelings 
clearly agree 



Often aware of 
subtle feeling 



Freedom to Interact 



Pupils are: Rarely free 
.1 



Pupils: Rarely show , 
self-contrdl 



Occasionally free 
3 



Self-Control 

Occasionally ' 
show self-control 



Generally free 
5 



Generally show 
self-control ' 



1 
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"Classroom. Global Ratings - Cont. " 

Jxteftt t;o which acHvi*ies having clear cognitive focus charattexize the 
clitssroom: : • / 

Rarely About 1/4 of About 1/2 of About 3/4 of Occur almost 
, ; occur thfi time' the time the time . constantly ' 

1 i'?' ... . 3 ■ 4 " . 5 . 

Overall . Emotional -Attitudinal Climate 

Hig^ily Positive most Neither positive "Negative - Highly 
positive 6f the time . nor negative Occasionally negative - 
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TEACHER PRACTICES OBSERVATION RECORD 



TOT 


1 


\ 

"1 


tit 


TEACHER PRACTICES 
A. NATURE OF THE SITUATION ' 


1 




i 




1. T occupies center of attention. 


1 






2. T makes p center of attention. 








J. T makes some i^iijg as a iMnu center of p's attention. 


1 

i 






k, . T makes doing something center of p*s attention. 




1 


5. has p spend tine waitlnqt watcninq, listening. 


■ 1— 






^. T has p l^artictpate actively. 








7. T remains dioof or detached from p*s activities. 








8. T joins or participates inp*s activities* 


; 1 ■ 






9. T discourages or prevents p from expressing self freel»Y, 










10. T encourages p to express self freely*' 


1 








r ' 

o. NATURt Oc'Tht PROBLtn 








11» T organizes', l^arni nq around Q posed by T. ^ 










12. T organizes learhing around p's own problem or Q; 






1 


13. T prevents situation which causes p doubt or perplexity. 










1^4. T'involves p^i n uncertain or incomplete situation. 










15. T steers p av<ay from "hard" Q or problem. 










16. T lead$ p to Q or proble^n which "stumps" him. 










17. T emphasizes* idealized, reassuring, on "pretty" aspects 
of topic. 










18. T emphasizes realistic, disconcerting, or "ugly" aspects 
of topic. 










19. T asks Q that p can answe\ only if he studied the 
1 esson . > 










20. T asks Q that i,s not readily answerable by study of 
lesson. 










C. DEVELOPMENT OF IDEAS 










21. T accepts only one answer *as being correct. 










22. T permits p to suggest additional or alternative 
f answer s» 










23«. T expects p to'conw up with answer T has In mind. 










2^.> T asks p to judge comparative Value of* answers or 
suggestions. 










25. - T expects p to "know" rather than to guess answer to Q. 










26. T encourages p to guess or hypothesize about the 
. iinknowh or untested. , 










27. T accepts only answers or Suggestions closely related 
to topic. 










28. T entertains even **wild" or far-fetched suggestion of p* 










29. T lets p "get by" with opinionated or stereotyped 
answer. 


* 


4 






30. T asks p to Support answer or opinion with evidence. 
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Teacher Practices Observation Record - Cent. 





1 


ii 


in 


0, USE OF SUBJECT MATTER 










31 • T collects «nd analyzes subiect matter for p. 










32, T has p make his own collection and analysis of 
subject matter. 










33 . T provides p wi th deta 1 If d facts and i nf ormat ion. 










3^^. T ^a$ p find detailed facts and information on his 
own. 










35. T relics heavily on textbook as source of information^ 




4- 






36, « T makes a wide range of information material available. 










37. T accepts and uses inaccurate i reformat i on. 










38. T helps p discover and correct factual errors and 
inaccuracies. 


.1 








39. T permits formation of misconceptions and over- 
qenera 1 izat ions . 










^40, T questions misconceptions, faulty logic, unwarranted 
conclus ions , 










, 'e. evaluation 










k\ , T passes judgment on p*s behavior or work. 










'♦2* T withholds judgment on p's behavior or work. 










^3* T stops p fxTom going ahead with plan which T knows 
wi II fail.' - ^ . „ 










M4, T encourages p to put His ideas to a test. 










US. T imncdiatel/ reinforces p's answer as "right" or 
" "wrong, " 










^46. T has p decide when Q,has been answered satisfactorily 










^7. T asks another p to give answer if one p fails to 
answer quickly. 










k6. T asks p to evaluate his own work. 










^3. T provides answer to p who seems confused or puzzled. 










30, T gives p time Co sit and thinks mu'l 1 things over. 










F. DIFFERENTIATION 










51, T has all p working at same task at same time. 










52. T has different p working at different tasks. 










53.^ T holds all p responsible for certain material to be 
'learned. 










S'*. T has p work independently on what concerns p, _ 










55. T^ evaluates work of alt p by a set standard, _ 










^56, T evaluates work of different p by different 
standards. 










G, MOTIVATION. CONTROL 










S7. T motivates pwith privileges, prizes, grades. _ 










58, T motivates p with intrinsic value of ideas or 

activity. _ 










59. T approaches subject matter in direct, business-like 
way. _ 










60, T approaches subject matter in indirect, informal wdj^^. 










61. T inpo.ses external disciplinary control on p. 










62. T encouraqes self-discipline on part of p. _ 
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^' ^ Classroom Descriptions 



^ Column 

1-3- ^ Deck No, ^ ^ ' ^ 

4-5 Program 

6-7 Teacher's Names 

8 Grade Level (OsK; l=Eat. First; 2=:Corvt First; 3=:2n,i)' 

9,10 Observer^ 01 Dr. Soar 05 Dee 09 Henry 13 Keith ^ 17 Hose 

11,12 Observer2 02 Mrs, Soar 06 Eileen 10 Jeff 14 Mnrfro 18 AVoyne - 

03 Barbie H. 07 Gene 11 John 15 Mary 

04 Barbara M. 08 Harriet 12 June 16 Pat 
13,14 ^ No. of Children Registerod 

15 ^^^^ No. of Adults 

16 largest pupil ethnic croup present .l=Negro 

17,18 Number 2=;Anp:lo«6 ^ 

19 Second largest pupil ethnic ^group present^ 3=Iadiaa 

20,21 Number > 4=Span1§h Amen' • an 

22 Third largest pupil ethnic group present *^5=0'ther 

23,24 Number 

25 Teacher ethnic pcrou p ^ 

26 Major aide otb:iic f^rou p *^ 

27 Number " 

Second aide ethnic p:rob p 

29 Number 

30 Sexes (1) Male (2) Female (3) Both 

Physical^ Arrangement - 

31 Rows - n[check) 1 i f checked 

32 Tables and rows (check^) • 0 if not^che<ked 

33 Small group tables (Check) ' 

34 ^ Number of reading centers 

35 Number of interest centers 

36 Size of Community (will be ^filled in later) 

37,38 School Hours: Daily to 

^39,i40 Meals L Snacks: Breakfast^ to ; Lunch to 

AM Snack to ; PM Snarkj _to 

41,.42 StrOctured Learning with Teacher (opening exercises, lessons, et<", ) 

43,44 Structured Learning wjthout Teacher (desk work, workbook, et , ) 

45,46 Unstructured .Time* (free play, recess, et*-,) 

Above 5 items have 2 columns; one decimal 

Example: 5 hrs 30 mln^S^S; 40. minutes=0, 7; 1^ minutes=0.3 

47,48,49,50 Size of Classroom ft. , x ^ ft, (total souare ft,) 

51 .-^ Carpet . 0 = none 2 = larf^e rug (1/3 area or more 

1 = small rufj 3 = wall to wall 

^ Soundproofing 0 = none; 1 = ves 
/ Number of years of pyovious school experience of the typical 
child in thd class- (include Headstnrt years) 

54 Number of years the teacher has had these same children h^r 

^ ■ class previously. (Onnot before this year; l=one year previous 

to this etc, ) , . V 

V Other Grades in this Classroom (Use grade code w. Column R) 
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B. INSTRUMENTATION .(continued) 
3. Data Collection Forms 



. • Student Data Collection Form (SDCF) 
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STUDENT DATA COLLECTION FORM 



1. Student's Name or Code 

2. " School District '_ 

3. ' School 



4. 'Date * . • ' 5. Grade 



6. Date of Birth ' month ; year 

7. Sex ED- male CH female 

8. Race dl Caucasian or White 

D Negro or Black 
I I Spanish Surname 
I I ' Oriental 
I I ' American Indian 

n Other (Specify) 

9. Is English a foreign language for this student? 

0 Yes □ No ' ^ ' ' 
10. Leyel of education of head of household 

Graduate from college 

1 I : Attended college x 
r~l Graduate from high sc'hool • 

I I Attended, but .did not graduate from high school 
s ~ O Finished 8th grade but did not attend high school 
I I Did hot finish 8th grade 



270 
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student Data. Col lection Form (2) , 

11. Occupational category of head of household 

! I Professional (Doctor, Lawyer, etc.) 

I I Business owner or manager- ' 

(31 White collar worker (clerk, salesperson) 

[~| Skilled worker: farm owner 

I I 'Unskilled, farm or service worker 

I I Unemployed 

12. Estimated family annual income 

[~| $15,000 and 6ver' 

Q Between $12,000 and 14,999 

[]] . Between $9,000 and if, 999 

I I Between $6,000 and. 8,999 - ^^ 

n Between $3,000 and 5,999 

O Under $3,000 

13. Standardized Test Measures 
(a) Name of L Q. Test 



Score 



(b) Name of Reading Achievement Test , 

^ Score -^pre) 

(c) Name of Math Achievement Test (post) 

Score ' . (pre) 

• . I (post) 



14. How many years experience has this child had in individualized 
instruction? ^^^...^ , - 

i 

years 

271 
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student Data Collection Fom (3) - ' • 

# 

15. How many days actually attended by this student? 

' days 

16. How many hours a day in reading/math? 

hours- Reading hours- Math 

r— ^ 

17. Educational opportunity = #days x #hours instruc./day » 

hours 
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b; instrumentation (continued) 
* 4. Test Inventories and Questionnaires 

§ My Class Inventory (MCI) 

§ How I Feel About Reading/Math (HIFAR/M) 

§ Teacher Questionnaire (TQ) 

§ 5urvey of Individualized Reading and Math 
Programs 

School Principal Questionnaire (SPQ) 

• Purdue Teacher Opinionaire (PTO) 

• Selected NAEP Reading "Items 

(a) Summary Data 

(b) Selected Iteifis 

• Selected NAEP Mathematics Itemsji 

(a) Selected Kems 

(b) Suimiary Data by Item 

(c) Scoring Directions Where Applicable 




These Sections 
are , 
Interleaved 
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MY CLASS INVENTORY (l) 



iny class 



r 




\ 




/ 
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MCI (2) ' . . ; v 



DIRECTIjOKS 



* Hiis, is not 'a test. ^. The jju^stions^ inside are^ to. find out what \ 
, vour clasis IS like. \ P^e^age answer all the questions. v • 
\ \ , . ' > . ' • ' " 

Eacft sentence; is 'meSrit to'- desirribe^ your class . If you, a^ree, with 
^ the sentence circle yes: » If you don't agre.e with the' Sentence.; ' 



circle no.' 



Examiile 



1. .Most children in the class are gQ.^d friends. . 

If you think that most children in the class l^ire ^;ood 

friends, circle the, yes- like this: ^ - 1 ' ' ' 

1. Most children in the class are good f rienc^i?'. ^ 

If you do not think that most children' in, the clfkss are 

good friends, circle the no like this: ^ ' * * 

-1; Most children in the class are^ gpod f-riends. 



Circle 

Your 
Answer 

fes No 



Yes) No 



4 t 



V*: 



eYes C'No^ 



Now\t\irn the page\ and answer all the questions about- your elas3^ 



ERic; 
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-4' . 



■4? ♦ f 



Circl6 
Your 
Ansyer 



No 



^ * / 1.' Thi^;; pupi Is ^en joy the ir s choolwqrk ih fay c las s . . Ye s 

6h±;ldretf ^&re alwfeys f ighti^ig'wi^h- each other. ' . Yes Na 
,3h'^ same people always do th^ best work in our class. Yes No 



/ ',4>* In' axii^^"clas& the work is hard^^tb .do. . ! \ • '^"^ - Yes • No 

• . . - " - . , - • ) ^ . ' 

',5.;\My best friends' are; in ray »Qlass^ : . % . ^ • - Yes^ No 

J:'^ 6. Some, of the. children ;Jnr O'ur^ class are main/ '- ' • Yejs : No 



7^. Most piiBils\ar.e please d^ith the^ -cljass; ' . / j • . ; tes 



No 



8. Children often race'^td 9ee who can flhifeh- first.. • ' " Yes N0 



• 4" 



9. Many children in the5 crass...play , torgetljeT afjter' - 



10. Mpst, children can;do J>hcir sthobl^^^ ,;Yes, . No ., 

^li/ Some 'pupils^ don't liker .the.^slass. ' "'^ ' - ^ ^ ^ 

12i. Most childreji^ajt^t .tjieir work ,to „be. bei:ter than o , ' \ ^ 



/ 1^.' Many cfelldr^n lil^ cdur clasis li1c6^'-to^f ighl ; - - . - - Y^s'-^ .^No ' 

r :J14V 0?j}y ttie ' smart*. people' CAtv Ac >^&rlc ia/op^r claVs. Yes No" 

[^,,->\:l5^.^Jt^[i-,jfoy^ everybody ^is' my jfTi^nd// " * * v. YeB !Nt?; * 

/ '" ' ' ' t , ' f^- ' * " ' . '~ ^ ^ ♦ ' • , il . ' " • J • 

- * -/^^'^ X'^"^: . « > 1 

• ■ ; W;- '7-64'...;, A " 

vf- . , \% . ;.. ,; < - ....... - * 



MCI (4) *. * , ' 



ft * 

4r ' 



J 

n 



\ ' ' Circle 

, - Your 

Answer 

16. Most of the children in injr clasa.enjoy schoO;l. - *^ Yes ' No 

4 17. Some pupils don*t like other pupils. ' \* ' 

• 18. Some ^pupils feel bad when they do, not do as well 

, - > . the others. ^ ^ Yes. No 



19. In my class I like to/<fork with others. * ' Yes 



No 



207 In oui^. class all the pupils know how to do , * 

tHeir. s.cho^olwork , • Yes No, 

.•.2i; Host children say the class is fun. Yes No 



22f Shn^ ^jfeople in my class are not my. friends. ^ ' Yes No 

23^, Wren, have secrets wi'th other children in ^ ' • ' 

^ .'thfe'^ilass: ' v- v-"^ ' Yes No. 

thtldreii Of t^.n find their work hard. ' Yejs No \ 

'>25..'^Mpst children 'don' t care who- finishes first, . Yes No 

26*; -.Sdme *cllildren -don' t like, other children . Yes No 

//^27^... Some;* pupils arie not happy in clasa. ^ , ' - Yes No 



J 28-. Ail of, the children' know each other" well. .\ ' Yes No 

\ . '. ' ■ ' > ' ' ' ' • 

V, 29. -Only the smart pupils can do their work. Yes No 

30,' Som^' pupils always try to d6<. their work better' 

■ than, the '.others J ^ -.^ ^ Yes •. No 
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31; Children seem to like *the class. 

32. Certain pupils always want . to have their \pwn way^ 

33. 'All pupils ill my class are close friends. 



Circle 
Your 
Ahs,wer 

Yes 

Yes 

Yes 



Nq' 
No 
No 



34. Many pupils in our class say that school ^is easy. 

35, In our class some pupils^ always want to do best^ 
36~. Some of the pupils, don ' t like the class.' 



Yes 
Yes 
Yes 



No 
No 
No 



37. ' Children ip our class fight a lot. 

38. All of the pupils in my class -like^one another. 

... ■ ' . • ^ . " ■ 

39. Some pupils always do better, than tlje rest 
of the class. * ' " • 



Yes 
Yes 

Yes 



No 
No 

No. 



40. Schoolwork. is hard to do. 



41. Certain pXipils don't like what, other pupils do. 

42., A few children in my class want to" be* first" 
all of the time. ' * * 



Yes No 
Yes No 

Yes No 



^ 43. 
44. 



Ihe class is fun';, 



Most of; the pupils in my class know how to 
do their work. 



45. Children in ou^ class like each other as friends. 



YeSj. No 

Yes . No 
Yes *n6 
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, ,HOH I FEEL ABOUT READIfJG/MATH - 

We would like you to ^[nswer the following questions 
so that you Can tell us how you feel aboiit Reading/Math. 
This ls not a test and there are no rtght or wrong answers. 
Just answer as best you can. ^ You can answer either "Yes", 
or "No". Let's do one for practice! 



I LIKE TO WATCH TELEVISION 


YES 


? 


NO 



If you like to watch television you'would circle "Yes"; 

if you never like to watch television you would c'ircle "No". 

♦ ' 

If you don't Icnow if you like to watch television, you would 

«. . * * 

circle "?". Make sure you'circle one of the choices. If 
your' answer is*"some times" ^ you have to decide whether it is 
more yes or more no . - 

I will read every question. to you as we go along. Mark 
your ^answers with a circle next to each question.* If you 
arg not sure what a question s^iys then raise your hand and . 
•I .will help you. ' --^ 

Make sure thai you. answer all of the questions. Do not 
skip any.' Whfen you are finished you should ^work quietly at 
your desk until evjryone^has finished. . 

. ^ . ' 
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HOW I FEEL ABOUT READING/MATH. 

^: I like to talk about Reading/Hath at homeT^ 

2. I think Reading/Math class is boring. 

3. I Tike to talk about what I find in Reading/Hath lessons. - 

4. I like to Read Stories/Do Hath Problems over again. 

5. I like Reading/Hath . , . 

6. I. like to help my classmates with Reading/Hath problems. 
7: I like to read Story/Math books. 

8. I like school tetter qn the days that I have Reading/Hath cl 

9. Reading/Math is not an important subject. 

10. When the other students talk about Reading/Math I want to ■ 
walk away. * 

JTj. I like to talk to teachers about Reading/Hath ." ' 

12. I would Tike to teach Reading/Hath if i were a teacJner. 

Twould like to buy a Story Book (Reading Only). 

I would like to buy 'a Math Book (Math Only). 

14. ^ wish.. that I didn't have to take Reading/Math . 

'15. Reading/Math is too much work. * 

16. I like to get Reading/Math books when I go to the library. 

331. I can't wait for Reading/Math class to be over. 

18. I like to Read/Do Math Probleriis' at Kome. 

19. Sometimes I Read/Do Math Problems at home just for fun. 

20. I like to ask questions about Reading/Math . ' - 

21. I like to Read Stories/Do Math Problems . 

22. Mike to talk to'my friends about Readfhg/Math . 

* ^ ' 280 
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HOW I FEEL ABOUT READING/MfTH 
. ' DESCRIPTION AND, SCORING OF THE QUESTIONNAIRE 

\ 

Description: • ' • 

This inventory consists of twenty- two statements derived 
from the "How I. Feel About School and Science" questionnairisK^ 
The iteois have, bgen altered slightly to give a mea'sure of the 
student's general attitude towards reading or matli%mati.cs. As. "^ 
wf%h the standard forms, the subject may answer "Yes"l "?", or ' 
"No" depending on how he feel s^out each statement.- ^ 

■ . Scoring: , 

For those itans which express a favorable attitude 
towards reading/math (Numbers 1, 3,. 4, 5, 6, 7, 8, 11, 12, 13, 
16, 18,- 19, 2a.-21, 22) Score: ■ , . , 

%,<^ One point for' "Yes" answers; .• ^ 

Two points for "f" answers; 
Three ptfints for "No" answers?^- " ' , 
For t-hose items-which express a negative attitude towards 
reading/math (Numbers 2, 10, 14, 15, 17) reverse the scoring 
values. Thus: 

Three points ^or "Yes"- answers; 
Two points foA^'?" answers; 
One point for ""Nb-" answws. ^ 
Subject test scores are determined by adding the number of 
points given for each question.. ^ 
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TEApHER qUESTIONNAIRt 
(TQ) 



1: School District 

2. School 

3. Date 



4. Teacher's Name . - ^ ; . 

5. . Program Type: [Zl Individualized dl Standardized 

6. Race: 7^^^ 

□ 'Caucasian or White 
'Negro or Wack 
Spanishv Surname 
O - Oriental 

American Indian 
• _ D Other (Specify) 



7: Sex: .Male []] ^ Female P] 

8. How .many years of teaching experience, including the present 
year I have you had in teaching compensatory education learners? 

One year or less 

Two or three years 



• □ 
□ 



Four or five/years 
Six through nine years 
Between 10 and 20 years . 
More than 20 years 
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9* How many pupils do you have in your class? ^\ pupils 

10. What i-s the name of the reading or ' math , program with which you 
are working? ^ ' ' . 
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Teacher Questionnaire (2) 




11. How many years have you been working with the above program? 

ED This is my first year 
D This is my- second year 
O This is my third year 
D Four or five years* 
'.<CZI More than five years 

12. What was the nature of the speci.al training or orientation 
you received in regard to implementing the above program? 
(Check all thaf*apply) 

D I received no special training 

D After school or weekend workshop,^. 

D Released- time workshop 

D Sumner workshop or institute 

tH • Individual instruction with supervised teaching 

CD College course 

^ D Other (Specify) _^ 



13. How adequately was your. special training in preparing you to 
^ implement the above program? 

CD Very adequate 

CD Moderately, adequate 

, D Fairly adecj^uate ^ ^ - ^ 

4 ' CD * Not at all adequate 
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Teacher Questionnaire (3) 
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14. Do you believe that the compensatory math or reading program 
you are worki\ng with is generally .worthwhile? 

Q Definitely- YES - 

Probably- YES 

I am undecided 
CD Probably- NO 

n Definitely- NO ^ 

15. How rtuch cooperative planning is there among principal and 
teacher in terms of program goals, improvement^ problenjs? 

I I A great deal 

I j A moderate amount 

None at all 

16. Would yoij strongly reconiDend the math or reading program,^ 
with which you .are working, to other teachers? ' 



V 



Reading 


Math 




□ 




Definitely--YES 




□ 


Probably—YES 




□ 


I am undecided 


□ 


■ □ 


Probably— NO 


□ 


□ 


Definitely --NO 



□ □ 



Question not applicable to me 
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Teacher Questional re (4) , 

17. If ^ou had your choice regarding curriculum 
for next year, would you choose to continue to. use 
the same reading or math program? 

Reading Math ^ 

Q Definitely-^YES 

* . O ^ Probably^S ' , 

|~| I 1 Tam undecided 

en ' C] Probably--NO 

[3 ' Q Definitely-NO 

I { Question not applicable to me 
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SURVEY OF INDIVIDUALIZED READING.AND MATH PROGRAMS 
• • SCHOOL PRINCIPAL QUESTKIlNNAIRE 
(SPQ)' ^ 



School Name 



School District 

Principars Name . 

Program Type (Individualized or Standardized) 



Directions : This questionnaire is intended to elicit information 
about your school and the students in youy? school. Information 
is..also elicited regarding your school district's policy regarding 
in.dlidduili2ation and the degree of support offered by parents and 
advisory groups. 

Answer all questions with reference to the current school 
year unless otherwise indicated. 



1. School enrollment this year (nmber of pupils). 
CI Less than 100 

Q 100-29? ^ . • . 

n '300-499 
" n 500-699 
d 700-899 
* GH 900 or more 

"2. Estimated pe^fcentage of tlwdents of the following racial or 
national origins. {Check only one box in each lettered row). 



0-10% 11-50%- 51 --90% 9>-100% 



(a) Caucasian ^or White 

(b) Negro or Black 

(c) Spanish surnamed 

(d) Oriental* 

{e)*American Indian 

(f) JDther (Specify) _^ 
/ 
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School Prtnctpal Questionnaire" (2) 



3. Estimated percentage of pupils whose head of household 

attained the following levels of -education. (Check onl^/*-^r'%>- 
one box in each, lettered row). 



11-50% "51-90% 91-100% 



(a) Attended college 

(b) Graduatea frm high school 
but did not attend college 

(c) Attended but did not 
graduate from high school 

,(d) Finished 8th grade but did 
not attend high school 

(e) Did not finish 8th grade 























— ^ 






/ 















4. Estimated percentage of school families in each of the following 
occupational categories. '(Check only^bne box In each lettered 
fow). 



0-10^^ n-50% 51-90% 91-100% 



(a) Professionals (Doctors, 
' ' Lawyers » etc J 

(b) Business owners or - 
managers ^ 

(c) White, collar workers 
(clerks, salespeople, 
etc.) 

(d) Skilleid workers f farrm 
owners 

(e) Unskilled,^ farm, or 
service workers 

(f) Unemployed • 
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School Principal Questionnaire (3) 




tmated percentage of school families that have each of 
following annual incomes. (Check only one box in each 
tered row;. 



(a) $12,000 and over 

(b) Between $9,000 and 11,999 

(c) Between $6,000 and 8,999 

(d) Between $3,000 arid 5,999 

(e) Under $3,000 



0-10% l]-50% 5}-m 91-100% 

































f 











6. About what percentage of the families of students participating 
' In math or reading instruction are represented as parent volun- 
teers to assist in some way in school? • 



□ 


0-5% 


□ 


21 


-40% 


□ 


6-10% 


□ 


41 


-70% 


□ 


11-20% 


□ 


71 


-100% 



About what percentage of the families of students at your 
school are represented at a typical meeting of the PTA or 
similar parent group? 



□ 
□ 



0-5% ■ 

6-10% 

11-20% 



□ 
□ 
□ 



21 -40% ' 
41 -70% 
71-100% 



8. What percentage of new programs or curricula used in your 
school originate from the following 'sources? 





None 


A few 


Several 


Many/Most 


Teacher Curriculum Committee(s) 


□ 


□ 


□ 


□ 


Community Pressures 


□ 


□ - 






Principal 


□ 


□ 




□ 


District or Area Office 


□ 


□ 




□ 


Curriculum Ccordinator(s) 


□ 


□ 


□ 


□ 




V-77 


2-89 








School Principal Questionnaire (4) 



9. How long have vou used this compensatory reading and/or 
math program(s) in- your school?. (Fill in names of programs) 




Program 1 Program 2 Program 1 Program 2 
One school year or less I I I 3 

More t^an 1 but less than 2 [ ' 
More than 2 but. less than 3 
Three or more years '~ 



s 

10. To what degree has your school district supported the imple- * 
mentation of your reading and/or math cpmpensatory education 
program(s)?- . 

» 

Not Sljghtly Moderately Very 

• Supportive Supportive Supportive Supportive 

(a) Monies for non- p— ) ^ 1 1 

professional' 

support staff 

(b) Released-tlwe • 
for professional 
staff 

(c) Monfes for add- ] 
itional pro-' 

fessional staff 

(d) Monies for ^ ' 
training work- 
shops* or 

institutes , t 

(e) Other , (Specify) ' ^ ; 




11, What is the average turnover rate 1n your school on a 
yearly basis? 
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- The Purdue Teacher Opim'onaire 
(Bentley, R", R. and Rempel, A. M. The PuFdue Teacher Questionnaire, 
Indiana, University Book Store, 1967. ) 

Purpose and Use - 

The Purdue Teacher Opinionaire is designed to provide a measure 
of teacher morale. ,Not only does the Opinionaire yield a total score 
indicating the general level of a teacher's morale, but it also 
provides meaningful sub-scores which break down morale into some of 
its dimensions. The ten categor*ies included are: (1) Teacher Rapport 
with Principal; '(2) Satisfaction with Teaching; (3) Rapport Among . 
Teachers; (4) Teacher Salary; (5) teacher Load; (6) Curriculum Issues; 
^) Teacher Status; (8) Community Support of Education; (9) School 
Facilities and Services; and (10) Community Pressures. 

The instrument can be useful to school adminfstrators, school 
staffs* and researchers who desire an objective and practical index of 
teacher morale in j)articular schools or school systems. Comparisons 
can be made among teachers when grouped by schools, grade levels, 
subject areas * ^enure status, etc. The Opinionaire provides specific 
and valid information about crucial problems and tensions which concern 
* the faculty and have an adverse^effect on their morale. Very basic to 
improving the level of morale is an adequate understanding and diagnosis 
of how teachers feel about their particular school situation. 

^ Directions for Administration 

The directions for completing the Purdue Teacher Opinionaire are 
given, on the cover page of the Opinionaire and are self-explanatory. No 
time limit is imposed; however, most teachers wilT complete the instrument 
in 20 to 30' minutes. In order to obtain valjd and reliable data, all re- 
sponses to the instrumenKsi^ld remain strictly confidential. 



THE PURDUE TEACHER OPINIONAIRE 

Prepsurec) by Ralph R. Bentley and Avemo M. Rempel 



TWs instrument is designed to provide you the opportunity to express your ppiiii^s about 
your work as a teacher and various school problems in your particular school situation. There 
arc no right or wrong responses, so do not hesitate to mark the statements framdy. 




FiD in the information below. You will notice that tHere is no place f^^ur name. Please 
do not record your name. All responses will be stjttly confidential, and result$.will be reported 

by groups only. DO NOT OMIT ANY ITEMS. 

^ 

School Date \ 



month day year 
Ag^ Sex Highest Degree Completed—^ 



DIRECTIONS FOR RECORDING RESPONSES ON OPINIONAIRE 

Read each statement carefully. Then indicate whether you agree, probably agree , probably 
disagree , or disagree with each statement. Mark your answers in the following manner: 

If you agree with the statement, circle "A**. , PA PD D 

If you are somewhat uncertain, but probably agree with the statement, ^.^^ 
circle •'PA" A dPA) PD D 



If you aro^mewhat uncertain, but ^probably disagree with the state- 
ment, circle "PD" i A PA tPD) D 



If you disagree with the statement, circle ^'D" A PA PD 
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1. Details, "red tape," and required reports absorb too much of my tinie A PA PD D 

2. The wdrk of individual faculty members is appreciated and commended by our 

principal „ A PA PD D 

3. Teachers feel free to criticize administrative policy at faculty meetings cajled by 

our principal , „ „ „ „ „ Ji PA PD* D 

4. The faculty feels that their ^ggestions pertainihg to salaried are adequately 

transmitted by the administration to the board of education 1...A PA PD D 

5. Our principal shows favoritism in his relations with the teachers in our school A PA PD D 

6. Teachers in this school are expected to do an unreasonable amount of record- 
keeping and clerical work „ „ „..'.A PA PD D 

7. My principal makes a real effort to maintain close contact with the faculty A PA PD D 

8^ Community demands upon the teacher's time are unreasonable ^ ^..Ji PA PD D 

• .9. I am satisfied with the policies under which pay raises are granted Ji PA PD D 

10. My teaching load is greater than that of most of the other teachers in our school... PA PD D 

11. The extra-curricular load of the teachers in our school is unreasonable - A PA PD D 

12. Our principal's leadership in faculty meetings challenges and stimulates our pro- 
fessional growth „ « „ ^ „ Ji pa' PD D 



13. My ^ching position gives me the social status in the community that I desire A PA PD D 

14. The number of hours a teacher must work is unreasonable ~ A PA PD D 

15. Teaching enables me to enjoy many of the material and cultural things Hike .-A PA PD D 

IS. My school provides me with adequate classroom supplies and equipment ^ .A PA PD D 

17. Our school has a well-balanced curriculum : Ji TA ' PD D 

18. There is a great deal of griping, arguing, taking sides, and feuding among onr 

teachers - ^ ^ ^ „ ^ ^ _,A PA PD D 

19. Teaching gives me a great deal of personal satisfaction ^ Ji PA PD D 

20. The curriculum of our school makes reasonable provision for student individual 

differences „ ^ ^ Ji PA PD D 

21. The procedures for obtaining materials and services are well defined and efficient..„A PA PD D 

22. Generally, teachers in our school do not take advantage of one another A PA PD D 

23. The teachers in our school cooperate with each other to achieve common^ per- 

aonal, and profcssiprtal objectives Ji PA PD D 

• * ^ P 9 3 Continue with item 24 on next page" 
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24. Teaching enables me to make my greatest contribution to society A PA I^D D 

25. The curriculum of our school is in need of major revisions A PA PD D 

26. I love to teach \ ......A PA PD D 

27. If I could plan my career again, I would <:hoose teaching Ji PA PD D 

28. llxperienced faculty members accept new and younger members as colleagues A PA PD t) 

29. I would recommend teaching as an occupation to students of high scholastic ability... .A PA PD D 

30. If I could earn as much money in another occupation, I would stop teaching A, PA PD D 

. <» — 

31. The school schedule places my classes at a disadvantage ^ Ji PA PD D 

^4 ■ *- 

32. Within the limits of financial resources, tlie school tries to follow a generous 

policy regarding fringe benefits, professional travel, professional study, etc ^...A PA PD D 

33. My principal makes my work easier and more pleasant.....^ ^ ^ ^ Ji PA PD D 

,34. Keeping up professionally is too much of a burden ^ ^ Ji PA PD D 

35. Our community makes its teachers feel as though .they are a' real part of the 

community ^ ^ ^ ^ PA PD D 

36. Salary policies are administered with fairness and justice « , A PA PD D 

37. Teaching affords me the security I want in* an occupation « ^ Ji PA PD D 

38. My school principal understands and recognistes good teaching procedures -A PA PD D 

39. Teachers clearly understand the policies governing salary increases....! ^ A PA PD . D 

40. My classes are used as a **dumping ground" for problem students Ji PA PD D 

41. The lines and methods of communication between teachers and the principal in 

our school are well developed and maintained : A PA PD D 

' 42. My teaching load in this school is unreasonable ^ „ 1.-...A PA PD D 

43. My principal shows a real interest in my department..,..*. » ^ ,Ji PA PD D 

44. Our principal promotes a sense of belonging among the teachers in our school Ji PA* PD D 

45. My heavy teaching load unduly restricts my nonprofessional activities A PA PD D 

46. I find my contacts with students, for the most p^t, highly satisfying and rewarding...-A PA PD D 

, 47, I feel that-i am an important part of tiiis school system ^ ^ ^ Ji PA PD'^ D 

48. The Competency of the teachers in our school compares favorably with that of 

teachers in other schools with which I am familiar ^ Ji PA PD D 



. , . o r^ >c Continue with item 49 on next page 
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49/ My school provides the teachers with adequate audio-visual aids and projection 


A 


PA 


pn 






▲ 

.A 


t 

PA* 


Pn 


Lf 




▲ * 


OA 




L) 




A 


PA 


Pn 


LJ 




A 

.A 


PA 


Pn 


D 






PA 


pn 


T\ 
\J 




A 


PA 


PD 


D 


56.' I a|n at a disadvantage professionally because other teachers are better prepared 


A 


PA 


irL) 


U 




A 


PA 


on 


u 


58, As far asl know, the other teachers think I am a good teacher - ^ 


A 


PA 


PD 


D 


59. Library facilities and resources are adequate for the grade or subject area Avhich 




PA 


on 


n . 


60. The "stress and strain" resulting from teaching makes teaching undesirable for me.. 


..A 


PA 


.PD 


D 


61* My principal is concerned' with the problems of the faculty and handles these 




PA 

rrL 


pn 


LI 


62. I do not hesitate to discuss any school problem with my principal 


A 


PA 


on 


U 


63. Teaching gives me the prestige I desire ^ ^ 


A 


PA 


PD 


D 


64. My teaching job enables me to provide a satisfactory standard of living for my 


A 


P A 


on 


U 


■d^t^ «*<M 1 111* Ill .l1 ^ • j1 1. 

65. The salary schedule in our school adequately recognizes teacher competency 


. A 


PA 


T%T\ 

PD 


D 


66. Most of the people in this conairiunity understand and appreciate good education... 


A 


PA 


PD 


D 


67. In my judgment, this community is a good place to raise a family 


A 


PA 


PD 


D 


68. This community respects its' teachers and treats them like professional persons. . . 


A 


PA 


PD 


D 


69. My principal acts as though he is interested in me and my problems 


A 


PA 


PD 


D 


70. My school principal supervises rather than "snoopervises** the teachers in our 




T> A 

PA 


PD 


D 


71. II is difficult for teachers to gain acceptance by the people in this community 


A 


PA 


PD 


D 


72. Teachers* meetings as now conducted by our principal waste the time and energy 


A 


PA 


PD 


D 



Continue with item 73 on next page 
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73. My principal has a reasonable understanding jof the problems connected with my 

teaching assignnxent - -A PA PD £^ 

74. I feel that my work is judged fairly by my principal 



75. Salaries paid in this school system compare favorably with salaries in other sys- 
tems with which I am familiar : : 



77. The cooperativeness of teachers in our school helps make my work more 
enjoyable 7. - - ^ ^ - - ^ - - 

78. My students regard me with respect and seem to have confidence in my profes- 
sional ability ..^ > ^ ^ ^ - ^ ^ ^ 

' 79. The purposes and objectives of the school cannot be achieved by the present cur- 
riculum - - - - _ - ^ 



80. The teachers in our school have a desirable influence on the values and attitudes 



82. Idy.students appreciate the help I give them with their school work.. 



PA 


PD 


D 


PA 


PD 


D 


PA 


PD 


D 


PA 


PD 


p 


PA 


PD 


D 


PA 


PD 


D 


PA 


PD 


D 


PA 


PD 


D 


PA 


PD 


D 


PA 


PD 


D 


PA 


PD 


D 



85. As a teacher in this community, my nonprofessional activities outside of school 





-A 


PA 


PD 


D 


86. As a teacher, I think I am competent as most other teachers 




PA 


PD 


D 


87. The teachers \*ith whom I work have high professional ethics » 


A 


PA 


PD 


D 



88. Our school curriculum does a good job of preparing students to become enlight- 

ened and competent citizens ^ J:. - - ^ Ji PA PD D 

89. I really enjoy working with my students ^ ^ ^ ^ PA PD D 

90. The teachers in our school ^ow a great deal of initiative and creativity in their 
teaching assignments - - - _ ^ ^ 



PA 


PD 


D 


PA 


PD 


D 


PA 


PD 


D 


PA 


PD 


D 



94. The people in this community, generally, have a sincere and wholehearted interest 

In the school system ^ ^ ^ ^ ^ Ji. PA^^¥t) D 

Continue v/ithjiem 95 on next page 
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95. Teachers ^eel free to go to the principal about problems of personal and group 

^If*^ ' : ; :.A PA PD 

96. ^. This community supports ethical procedures regarding the apF>ointment and 

rMppointment of members of the teaching staff »^ A PA PD 

r 

97. This community is willing to support a good program of education A PA 

98. Our community expects the teachers to participate in too many social activities A PA pb 

99. Community pressures prevent me from doing my best as a teacher A PA PD 

100. I am well satisfied with my present teaching position : A PA PD 



» 
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rog* ? 



o $ 2 



1. D-5 

2. - A - 1 

3. A - 1 

4. A-4 

'5.. Dr 1 

6.. D-5 

7. A - 1 

8. D-5 \ 

9. A - 4 

10. D-.5 

11. D- 5 

12. A - 1. 

13. A - 7 

14. D-5 

15. A - 7 

16. A - 9 

17. A - 6 

18. D-3y 

19. A - 2 

20. A - 6 

21. A - 9 

22. A - 3 

23. A- 3 . 
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24. 


A - 


2 
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A - 1 






49. 


A - 


9 


A- 1 


25. 


D - 


6 
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. A - 


2 


A- 1 






26. 


A- 


2 - 
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A - 8 








51. 


A - 


2 








27. 


A - 


2 


52. 


A - 


3 


\75. 


A- 4 


97. 


A - 8 


28. 


A - 


3 


53. 


A - 


3 


■ V 


• D- 2 


98: 


D-10 


29. 


A - 


2 


51. 


D- 


3 


77^ 


A- 3 


99.' 


D-10 


30. 


D- 


2 


55. 


A - 


3 






100. 


A-2 


31. 


D - 


5 








78. 


\ A-2 







32. . A - 4 

33. A - 1 

34. D-5 

35. A - 7 

36. A - 4 

37. A - 7 

38. A - 1 

39. -A - 4 

40. D -'5 




48. A - 3 



56. D - 2 

57. A -%9 

58. A-2 

59. A - 9 

60. D - 2 



61. - A 

62. A 



63. A - 7 



"65. 
66. 
67. 
68. 
69. 



A 

A 

A 

A- 

A 

A 



7 
4 
8 
8 
7 
1 



70. A- 1 ^ 

71. D- 7 - 
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D- 1 



79. 



80. 



D-6 



. \ 



81. D\-10 



82. Ai^ 2 



83. 
84. 



A-\2 



A-3 



85. D-710 



86. 



A-2 



87. A-3 

88. A - 6 

89. A-2 

90. A-3 

91. A -10 
■92. A - 1 

93. A - 1 

94. A - 8 



SCORING DIRECTIONS: PURDUE TEACHER OPINIONAIRE 



1. Tear yellow Scoring Key into five (5), strips, one for 
each page of the test* 



2. tine up appropriate strip to the left of the -answer colunms* 

3. On each page, score the "Red D" Items first. Oa page^2, fot 

exanpley, these are Items: Sy 6y By lOy lly 14 and 18. 



**Red D" Items are scored- as follows: ; ~ 

®- 1 

@ - 2 . ^ 
@ - 3 

Write down the appropriate item score f ot each Item to the right of 
the answer column* Do not record anything other than a 1, 2, 3 
or 4. Jf none of the answers has been circled, put a dash ( — ) 
next to the item. 

5. After scording the "Red D" , score the "Black A" ones. 

"Black A" Items are scoted as follows : 

* .0) - 4 . ^ 

ig^ - 3 
@ - 2 
<S>- 1 

6* Again, wrlt,e down the appropriate item score for each item to the 

right of the answer column. Do not record anything other than 
al, 2, 3or4. If none of the answers has been circled, put 



dash ( — ) next to the item. 
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purdUI teachejbu-opinionaire: directions for coding ^ ^ 

Use oneitransfer fonn for every 'six teachers in a given school. 

Each teacher will take two lines (cards). The Header information 
^ <Columns 1 - 14) is repeated on both lines. Line 1 will cover 
Items 1 - 60; Line 2 will cover Items 61 - 100. 

Columi;^ 1- 5: School Code . This is the five (5) digit nximber written 
In pencil under t^e words, /'Prepared by Ralph- R. Bentley.../* on 
each test. 

Columns 6-7: Teacher Number . This is the one (1) or two (2)^ digit 
number written after the School Code Number. Any number between 
1 and 9 must be coded with a '*0" first; i.e^. 01, 02, 07, 

Polqmn 8: Card Number . Each teacher will have two cards. Write in "1" for th 
for the- first card (on the first line per teacher) and "2" for the 
second card (on the second line per teacher). 

Columns 9' - 10: Age. Copy the actual age given on the Opinionaire. If 
it has been omitted, code a "00" in the two cottimns. 

Column 11: Sex. Code a "1" for Female and a "2" for Male. If this infor- 
mation has been .omitted, code a "0" in this column. 

Columns 12 - 13: Years Teaching . This is the first of the two numbers written 
in below the words, "Highest Degree Completed", on each test. Copy the 
number as it is, remembering to put a "0" before any one-place digit; 
i.e. 01, 05, 09. 

Column 14: Years Teaching- IP I * This is the one (1) digit number written 
in after the number for Years Teaching. Copy as is. Most Control 
School teachers will have a "0^*. 

♦ 

Colunnis 15.- 20. ■( Leave BLANK. 

Columns 21-80, Line 1. Record scores for Items 1 - 60i These are all 
one-digit, and can be only a 1, 2, 3 or 4. | 

Coluifiis 21-60, Line 2. Record scores for Items 61 - 100. [ 

Any items for which no answer has been circled and ^r$, therefoore, unscored, 
should be "coded" with a dash (-)• } 
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SELECTED NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS (NAEP) 

Reading Items ^ 

We have selected released NAEP items for which no copy- ' 
right release is requested. Items were selected such that the 
national per'tjaptage of nine year olds responding correctly was 

85% and the percentageoqf inner city (low^metroy pupi^ls 
was >^ 40%. Items from each pf five themes were selected. * 
Item numbers, national and low 'metro percentages, mode of 
/ administering and timp of administering for the items selected 
are shown below for each^ thertne. Total testing time is 25 minutes. 
A Summary Data for Selected Reading Items 
Theme 1 ; Understanding words and word relationships 

Item # National % Inner city Mode of Admin . 



RIOIOI 



95.2 



,88.7 1) Directions-tape 
recorded 

2) Stem read by 
respondent 

3) Response-written 
* by respondent 



Time of Admin. 

7 

, ^3/4 min. - 



R10403 


91.6 . 


81.8 


1) Directions tape 
recorded 

2) Stem read by re- 
spondent 










3) Response-written 
by respondent 


1 min. 


R10601- 


85.3 


' 63.0 


ii 


. 1 1/2 min. 


R10901 


88.1 


^.8 


li 


1 1/4 .min. 


RT2101 


92.6 


79.8; 


II 


1, min. 



5.5 min. 



< 
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Sunmary Data - 


NAEP Reading Items (continued).." 




' Theme 2: 


Graphic Materials ' 






. Item # National % Inner City % 


L*iUvic u 1 r\mii III* 


Time of Admin 


'R2O301 ' 8.75 


'68.0 


1) Directions tape 
• recorded 


1 1/4'min. 


R2O601 88.8 


•80.2 ^ 


^ tt 


1 min. y 


R20801. 87.3 






1 min. . 


R21O01 85.3 


^ Do. 3 


II 


1 Mi mia. 


R21201 .97*0 


94.1 


• II 


3/4 min. 


R2140a '85.4 


68.5 


11 


3 min. 



Theme 3: Written Directions 
Item # N'ational % Inner city % Mode of'Adpiin . 
R303oi 93.3 ' ^ 83-. 8 



8.5 min, 



Time of Admin* 




1) Directions-tape 
. recorded 

2) Stem-read by 
" r-espondent 

'3) Response-written ^ 
by respondent 



^ Theme 4: Reference 'Material 

Item # National % ^ Inner city % Mode of Actmin . 

R41203 91.3 '81.2 ^1) Directions-reacl 

by interviewer 

/ ' ' 2) Stem-read by 
. \ Interviewer 

3) Response-oral by 
" respondent .written 
'\ , by interviewer- 



3 min. 



Time of Admin . 
6 min. 



R41205 '92.9 



87.7 



5' min, 
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Sumnary Data - NAEP Reading Items (continued) 

Theme 5: Gleaning Significant Facts from Passages 

Item # National % Inner city % Mode of Admin . 

R52001 ' 85.7 74.0 ' 1) Directions-tape 

recorded 

2) Stem- read by 
respondent 

3) Response-written 
by resppndent - 

Theme. 6: Drawing Inferences 

Item # National % Inner c4ty % Mode of Admin . 

R71401 ' 86.0 -65.7 1 ) Directions-tape- < 

recorded 

- ' 2)* Stem-read by 

respondent 

. , 3) Response-written 
by respondent 



Time of Admin. 



1 min. 



Time of Admin. 



1 min. 
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Selected fclAEP Reading Items 



ACS tlVSLt 
miASI MOi 

OftJKTIVti 



XDEMTXCAX. OR SIMILXK 
Moox or AMIMXmXnONt 



TZNX or ACMINISTltATIOMt 
COr^XICRT MQUIRXNZlfTt 



» 

1^11 
X KXk 

I.. UndcrsUndlnf Wor4 K««nlAf« 

A, < In JsoUt^loa \ 

Af« It >-5 ' ^ 

Group 

Dlractlont - r«p« r«eord«4 
StM - Mad by ratpondMt 
M«pon«« - Written by r«tpood«at , 

9/4 minuU ^ 

Ho 



H«r« ar* plcturat of four door* you iilfht find In • •cIkmI. 
rill In th« oval under th« door wh«r« ybu aifht fo for luaeh. 




CZ> I don't know. 



ACt LSVILi 




9 


13 


neizAsi NOt 


A. 


10401 


10401 




B. 


10402 


10403 




b. 


10409* 


10403* 


rACXAGI-CXUICISK NOSt 




3-l( 


7-14 


OMCCTXVti 




XXI B3 





Kaad th« ••nt«nc«« and do what thay tall you to do. 



If you hava EVEK vlaltad tHa Moon, fill In th« ov«l 
Kara. 



IDENTICAL OX tINILAK 
rASSAOSii 

HODX or AnilNISTXATXONt 



TIME Or^ AONINISTMTXONt 
COmiCHT R^UXMMXNTt 



1 

Moo* 



Undsratandlng Word Maanlnga 
>. In Contaxt 



era up ^ 
Dlroctlona - Tap* racordad 
Sta« • lt«atf by raapondaiit 
Maponaa - ir:^ttan by raapondant 

A9a )t 1 lilnuta ' 
Af* 13t 3/4 iiinut« 



If you hav^ NEVER vlaltad tha Hoon. fill In tha oval 
Kara. 



*Spactal d«rlv*d Talia«t 



00 - No raapona* 

10 
20 
21 



Navar ba«n to tha Noon 
tvar b««n to th« Hoon (Incorract) 
tvar and Navar baan to tha Moon (Incorrect) 
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Selected NAEP Reading Items 



jnOf UViLt 
KfXXAlX not 
yACMCS-IXBUCIte NOSi 
OiJ«CTlVtt 

vant 

tOOnXCAt OA IXMIUUI 
PASSACESt 

*M0Of or ACMimSTIUTIOH: 
TINE or AtMXNimUlTIOIIt 



> n 

101 io< 
)-U 4-S 
X » 

I. tJnd«rstmn4in« Mord N««nlAft 
B. In CoiiUxt 



Croop 

Dir«ction« - T«p« r«cord*4 
ttM - M«d by r«tpond«nt 
ftMpont* - ifrltt«n ^ mpondtnt 

1 1/2 «lnut«t 
Xf* I3t 3/4 minut* 



F*opl« vho>ruji zoo* toMtlM* put slqn* on snlMl c«g«« to^ 
t«ll wh«t th« anlMlt tttm Hk% or vh«r« th«y com froa. If 
you w«nt to c^zoo and cav thtM four sign* on dtff«r«nt e«9«t« 
which oo« would tall you that thara it a dan9troua.anl»al 
liulda tb« ca^a? Pill In tha oval baalda tha correct algn. 



h ont of tK« 
tmtilUit an^mali 
found In Am«rlca. 



Insld«'thli cag« 
It an •xtr«m«ty 
Itrocloti bnimal. 



InsMt this cag« 

U an'onlmgl 
that sUtpi oil 
Iht tlm». 



Inild« thU caa« 
It a rart typ« of 
•agio- ono of 
tho fow loft in « 
tho world. 



ACS LIVELi 
ftOZASt WOt 
fACXACZ-KXZICISS NOt 



XDKNTXCAL Oil SXMXLAX 
PASSAGES t 



NODS OP ADMXNIITIUTIONt 



TXNE or AXMXNItTMTIOIti 
COPYiaCVT KEQUIREMZirTt 



5-2 

I Sib ^ 

1. Ondaratandlng Word Kaaninft 
In- Contaxt 



Croup ^ 

Diractiona - Tapa racordad 
S€aa - Mad ^ raapondant ^ 
Kaaponaa - Writ tan by raapondant 

1 1/4 Hlnutat 

1^, 



ERIC 



Coatplata tha aantanca with tha worda that^ka tha MOST tanaa. 
Tha boy wantad 



a ^av- ball . 
O un4ar dlrmar. * 
O roda hit bilfa. 

to tha circus. 

atoppad raining. 

O 'x don*t know, 
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Selected NAEP Reading Items 



umi 

tXUASI liOi 
OWCCTXVSt 



lOQlTICXL OX tIMXLJUl 
NODS Of AOMINXSTKATXONi 



tOa or ASMINISTKATXOVi 



9 1) 

121 121 ' ^ 

}-10 12-9 
I hit 

}. rollowinf Krltt«n Direction* * 
9. C«rryln9 out Wrltttn 
" Directions 



Group 

Dlr^tlon* - Tap* rtcordvd 

•t«B - RMd by r*»pond«nt 

Has pons* - Wrltttn by raipooiaot 



X coapound vord !■ • vord vhTch Is Mda by joining tvo words 
tOf«th«r. *rill in th« oval b«iid« th« coapound word. 



Aft 9t 1 minut* 
*g# 13 1 y jU.nut« 



ACROBAT 



C, I A S S B 0 0 n 



S E P'A R A T E " 



S U ttn-E R 



9 Z don*t know. 



ACS Lsy^t 
nzhzxst aot 

PACXACC-eXtMCXfV Mt 
OSJCCTXWt 



XOCNTICAL on SINItAJl 



HOOe or ADMIN XSTIUTXONt 



TIME or ^AOMINXSTRATIONt 
COFYRIGirr RtQUXMMniTi 



9 

203 
IV A 



13 

203 

13-S. 



2. RMdinf tnd Visual Aid* 
<• A. Xntarptatinf Drawing* and 
ricturoa 



Croup , 

Direct iona - Tap« r«cord*d 
BtM - Mad by r«apond«nt 
RaaponM - Written by raapondsnt- 

Afa 9t 1 1/4 ■iautaa 
A^a 13 f I A ttlnuta 




Look at tha pictura and fill in tha oval b«aid« tha avntanca which 
tall* ICST what tha drawing ahow*., 



# « 



Tha fiah haa alraady aatan th* worm. 

CI> Tha wom ia^probabl^ not on a hook and lin«. 

4» Thar fiali Idoka aa^if hm ifr^foirfg to eat tha wopa. 

Tha fiah ia waiting for tha worn to- b« put 
in w^tar, ' 

I don't kriow. 



ERLC 
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" 'Selected NAEP Reading Items' 



XZLZMt NO: 

rACKAce-exeiicjsE vosi 
objeCtxvz I 

TMZMSi 



XOEMTXCAL OK SXMflAX 
MODE or AOHXNXSTMTXbH: 



TIME or ADMINISTIUTXON I 



' ♦ * . X3 

2oi' 204 

3-2 3-15 

I tXc / ' • 

2. R««din9 and Vitual Aid* 

A. Xnt«rpr«tin9 Oravinqt and 
Picturtt 4 ; 

Hon* * ' *• 



Group • < - 

Oir«ctioh» - ^ap« rtcordtd 
ttm - M«d by respondent 
Maponsa - Written by raapondant 



A^a 9: 3/4 ninuta 
Xqa X3: 3/4 ainuta 




^4 




Look lit th«^p4ctura;and fill in tha oval basida tha aantanca 

vhich tal^s iE5T what tha drawin* abbws. 

r 

<Z> ,A aifn is hAnging.by tha door. 
A aign is hanging on tha door. 
A aign is hanging ovar tha door. 
A aign is hanging naar tha door. 



I don't know. 



AfiC I.KVXLt 
JtZLCASE NOt 
rACXAC«-eX£JtCXtE NOSt 

OMEcrmi 

TKEXKi J * 



tDOfTXCAL OK BXMXLAll 
fhSthCUi 

MODE Of XOKZNXSTllA^XCKi 



'TXMe or AOMINISTXATXON: 
CQrYKXCVr REOUXttKEUTi 



9 

201 
4-1 

I Clb 
2 



13 

201 

1-5 



Raadinq and Visual Aids 
i. Kaadinf Signa and L«b«la 



Mona 



Group 

Oiractions - Tap* racord«4 " 
Stan - Itaad by raapondant 
Rasponaa - Writtan by raapondant 

A^a 9i 1 ainuta 
K§» 13 1 3/4 Bintfta 



^SoM road signs tall paopla who ara driving cars what to do. 

Othvr signs tall p*opV« who ara'walking vhat io do« 
* 1 

Xf you ara walking, which jiign tails you what to do7 rill 
in tha oval basida tiia corract aign. 



SPEED LIMIT 



20 

wuxfotvom 
ON SCHOOL lun 

mum 

cmminm 

nttscvT 



LEnTUM 
AOOWEO 
FtOMCCNTEK 
LANE ONLY 



IJEDESnUANS 

^ USE 

cmswAu 



MAIN STREET 

EXIT ON 
THROU^HWAY 
.500 YARDS 
AHEAD 

KEEP RI6HT 



EMC 
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Selected KAEP Reading Items 



;t Lcmt 

\^JtASt NOt 

hCMCX-SXBJCISt NOSt 

uccnvst 



oarrxcAL ok siMiuut 



COS or^ONINISTMTXONt 



XXS or AONXNXtTMTXOMi 
umtXCXT KIOUXKZKZKTi 



310 

X »3 

3 



310 
C-4 



U44in« tnd Visual Aid* 
A« Xnttryrating Drtwin?* and 
rictur«s 



Group 

Directions - tkp* rtcordsd 
tt«« - lta«d by raspondant 
MaponM - Krittan^by raapondant 



Af« 13 1 



1 1/3 Binutaa 
3/4 ninuta 




Look at tha pictura and-fllianJtha-oval^tida tba aantanca 
wbich talla BEST what:th« drawl nf'ahovr. * 

0> Tha boy hai t3fO-4ofa-oa-t laaah; _ 

0> Tha boy ta^wftkinf-bahind hia 309. 

^ Tha dog oii_tI«riatih;h«a-apota on^it, 

CD' Tha' dog.;tittinf-down ^aa apotajm-At. 



0> IJon'^knw, 



AGE UVILt 
ftSLEAJt NOt 
rACKACt-tXEMCISS lK>t 
OBJeCTIVKt 



XOCNTXCAL OK tXMXLAK 
PASSAGES I 



MODE or AOHXNXSTKAtXONt 



TXME or AOMtNXSTKATIONt 
COPYKXGKT KEQUXREMSNTi 



9 

313 
T-l 
I A3 

3. Madinf and Viaual Aids 

' ». Kaiding Sign* and Labala 



Group 

Diractiona - Tapa racordad 
StaM * Kaad by raspondant 
Raaponsa - Hrittan by raapondant 

3/4 ainuta" 

HO 



PilL in tha oval basida, tha sign that a boy Might look for 
tf ha naadad to taka a bua hoaw* 
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G I 4aa*t kaa«. 



Selett*6d NAEP Reading Items 



KOt UCVELi 
fttLEMS NOi 



FACKAce-Bxencxsi noes 





) 


13 


17 


X. 


21401 


21401 


21401 


1. 


21403 


21402 


21402 


c. 


21403 


21403 


21403 


D. 


2U04 


.21404 


21404 


E. 


2140? 


2140S 


21405 


b. 


21 cot 


2140( 


3140i 






10-3 





Look At th« road Mp and raad aach tanttnca c«r«(ully. If vtut thm 
MAtttnct a^yi i« tru«, fill in th« oval b«aida •Trua.« If what thm 
c«nt«nc« aayi ia not tru«, fill in tha oval bMid* Talaa." If you 
can't dacida if tha aantanca ia trua or falaa, fill in tha oval baai4a 
•l don't know.' - " _ 



OWECTIVtl 


IXX C4(3) 


1 


3 


niDiit 


2. Madinq and Viaual Xida 

C. lUadlnf Charta« Kapa and 
Craj^ 




IIOKXVIOVM , 

0 W-4-1 


IDCVrXCAL OR SXNXIAK 


Nona 




W » 


MODE or AOMXMXStMTXOHl 


Croup 

Diractiona - Tapa racor4«d 
Sta* - Mad ky raapondant 
Maponaa - Writtan by raapondant 


0 ' i 




TIME or ASNXNZSTKATIOHi 


A9« )i 3 alnutaa 
' kqm 13 1 3 ainutaa 
kqm 17 1 2 3/4 ainutaa 




' » • 1. 

A I 

* \ 1 


COmXCUT UQUXREMCfTti 


No 




• ./C^ \ 












taumJ 
cnrrf 











(continuad on naxt pafa) 



ACt LSVILi 






13 


miASS HOt 


K. 


3O301 


30301 




B. 


3O302 


30302 




C. 


3O303 


30303 




D. 


3O304 


30304 






3O30S 


30305 


rACXACt-IXXnCXSE NOtt 




1-17 


•*13 


OUtCTIVIt 




XZI B2 





XDEMTXCAL OR f XNXLAR 
PASfACESi 

¥KIOZ or ACHlMIITRATIOIir 



TXKE OP AONXHXSTRATIONi 

© - / 

COrYKXGHT R£QUXR£MIXri 



Pollowihf Writtan Diractiona 
%. carryinf out Writtan 
Diractiona 



Oroup 

* Diractiona - Tapa racofdad >^ 
Itaa - Ra»d by raapondant 
Rtaponta • Writtan by raapondant 



JVga %t 
A^a I3i 



3 Ainutaa 
2 liinutaa 



To tha ri9ht you will aat four diractiona which you ara ta 
follow. Do ai Many a a you can In tha tUta you ara f iv«n. 




», Mil* tiM w*r< «« ua* Um. 
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•Selected NAEP Reading Items 



Act UVILi 
KttlASt NOt 



fAcxAcc-mxcxte Nost 

0»JKTXV8i 



lOINTICAL OR SIMILAR 
MODE or ACHXNXSTMTXONi 



TXKK or AONXKXSTRATXONt 



COrYRXOHT RCOOXMMENTt 





» 


1) 


17 


A 


1. 


4l20i 


.4i201(t} 


41201 


41201 


2. 


41262 


41202(*) 


41202 


41202 


A. 


41203 


41203(*) 


4120) 


4120) 


». 


41204 


41204(t) 


41204 


41204 


C. 


4120S 


4120S(s) 


41205 


41205 


0. 


4120€ 


4120l(t) 


4120C 


4120( 


z. 


41207 


41207(s) 


41207 


41207 


b. 


4120S 


4120l(s) 


41201 


412«| 




10*2 


lS-2 


12»2 


2-r 



XXX C)(l), X Al 

XXX c)(n. X Au 

XXX C)(l), X AU 
XXX C)(l). X AU - 
^XZX C)(l)v^ I Al 

4. Kaadin^ and JK«f«r«nc« HAtarlaU 
• « Us* ot ll«f«r«nc« H4t«riala 

Noo« 



Xndividual 

Dir*ctton» - KMd by Int«rvi«v«r 
Stm - Ksad by Int«rvi«v«r 
Katj^ns* - Or*l by respondent 

WritMn by Xntarvi<v«r 



Af« 9t 

Aqs l)i 
Aqt 17i 

Adultt 

No 



( iiinQt«s 
( Minutes 
4 Minutes 
4 Minutes 



(Give dictionary to ttodent. 

Atk the firet tvo quoetione and record the Min points of hie responees.) 



rirtt esk: -What is s dictionary?'* 



•(Dictionary I JA book that tells you what words Mftn.* 
'A book that tells you how to use words. *J( 

Then eski "Whet doee ths word 'define' «ean7"*^ 



••(Define: "To 9ive the fManin9 of words," or 
'To tsU what the word Mane.") 



(It the atudent defines both words correctly, reed A-E to his end recor4. 
his enswere, evsn if he cannot do eoiM pr ell of the teeke. tf he c«n- 
nol define on« or both of the words, 9ivi his^the definition (e) , uein^ 
ths dictionary and ehowinf hie a word and its definition if neceeeery. 
then contifiue with A. 

* 

If he answers A, either correctly or incorrectly, continus with tt 
l\e cennot ^nswer A, 9ive his the definition!*) once sore and repeat A. 
If he then answers A, either correctly or incorrectly, continue with »^t. 
If he stops working, encourage his to continue, if he etill cennot 
answer A, diecontinue. ^xplein the situation in A, and 90 to the next 
exerciee. ) 



•A. open the book to pafe and tell mm any three words 
definsd on that pafe. 

1, /7yy..y. " 



2. /^^^>^ .^^ . 



/ 



i. Tell se the last word defined irv the "PVeection 

Pi^^/^/. 1 

C. Tell se the nusbsr of the laet pafe in the dictionary. 

^ • 



q. Tell M the nusbsr of the pe^e on which you can find ^t 
whet the word *b«ke" seene. 

Y / ' 

B. Ten se the word which is defined juet before ths word 
'house' and the word which ie ds fined juet ifter the 
word "houee." 



<7>.:X t4fore_ 

efter^ y/.. .^>^ 



*Other Acceptable Reeponsee , 

A. allifator. elliteration, allocete, e Hot, ellow, ellowence, 
. alloy, 4jl ri^ht, ellround,* al Itpice , allude, allure, ellution. 
aluMinus. 4lly, else sater, elsiqhty, al»ond. alMOst. alse, 
a««e house, aloft, elohe, along, alongside, aloof, aloud, 
alp. elpece. alpha, alphabet 



ERIC 
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Selected NAEP Reading Items 



ZZ UVXti 

ILCAJI NOi 

ACKAGe>kx£]ICXfl NOt 

WCCTZVti 

VEKZt 



£>C!*TXCAl OK f UULXIt gg^ 
ODE or AMlNXSTIArXORt 



IHE or AOMZHXfTKATXONt 
ory«lGOT RZOOXRZMXlfTi 



1-3 

X Clk , »- 

5. M«4in9 for fi^nific^At ractt 
A. Itvcogniting Mc^ual 
Xnformatioii \ 

Af« •« 2-4, $-1, «-$, 



Croup 

Direct iona - T«p« r«cord«4 
Stw * to*d by refpond«nt 
totpons* - Written by re»pond«at 

1 sinut* 

Mo 



K«U th« ttory «nd co»pltt« th« ttnttact which follow* it» 

Th« wind pu•M^<l th« bo«t farther and ftrthor out to m«. Xt 
•t«rt*d to rain aM Um f 09 fr«w thick, Th« boy and- hit fkth«r 
v«r« lost tt Ml. 

Tht w««tK*r w«« 

C4la, 
<=> dry, 

tuimy. 
• w«t. 



0> X don't know. 



V; 



' ACK UVtLt 
PACXAGI-KXZRCXfK HOt 

outer xvzi 



IDBNTXCAL OK f INXLM 



KOOZ or AONXHXITMTXOHl 



TIME or AOHXNXfTWiTXOIti 
COrYKXCIT KZOUIKZMSIfTi 



9 

714 . 

5-1 

X Clb 

7. toadin^ and Drawinf Xnferoneee 
A. Drawing Xnfar«nct« fron 
Information Oivon 

A«« ft 2>4, C^$« l>2, f-< 

Croup 

Dirtctione - Tap« rocordod 
Stw - K«ad by rttpondtnt 
Mipont* - Written by reepondant 

1 Kinuta 

No 



Raad tha eto/ry and anawar tha qua'ation which follova iti 

lha wind puaKad tha boat farther and farther out to eca«' xt 
•tartad to .i-ain and tha fog frew thick, tha boy and hit father 
ware loet ^at aaa. y : 

At laaat how aany people wata\{n tha boat? 



O rour 
0> riva 

O X don't know^ 



\ 
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SELECTED NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS (NAEP) 
' /^Mathematics Items 

Preliminary ex^rci^ Wta supplied by the NAEP were 
examined to identify mathematics exercises with the same 
characteristics as tfiose selected for Reading, i.e., NAEP 
items for which no copyright release is requested, and for'w'hich 
the- national percentage of 9 years old, responding' correctly 
was 85% and percentage of inner city (low metro) pupils was 
40%, The criteria for the national group were found to be 
too restrictive. •Successive passes through the data were made • 
lovyering the criteria for the tiational group by intervals, of 5 
percentage points. The set of items finally accepted were 
selected from four of the fifteen content areas tested. They 
are: . - , ' 

A, Number of Numeration Concepts 
- B. . Properties of Numbers and Operations 

C. Arithmetic Computation ' 

E. Estimation and Measurement 

H. Equations and Inequalities ' - r 

K. Geometry 

We have included for each exercise a page showing the 
exercise itself and a documentation page showing content area, 
objective, exercise .type, scoring type, administration mode, age 
group, national- P-value and tijnfng (in seconds). Total test time 
for the 24 items. is 20 minutes. 

y-ioo ' 



\ . ^ ■Selected NAEP Mathematics Items 
^ What digit is in the tens place in 4,263? 

C3 2 
CZ) 3 

CD 4 ■ 

♦ 

<±> I don't know. 



RHH! 
OBU 1 




DO NOT CONTINUE 
UNTIL, TOLD TO DO SO. 



Selected NAEP Mathematics Items 



Report I: 
Ni^BP »: 

Content Area: 
Oblectitre: 



BA01 

J^11001-1 



NOHBBR AND NttRESATION CONCEPTS 

I. -Recall and/or Recoqnition of Definitions, 
Facts, -and Symbols..- 



Exercise Type: 
Scoring Type; 
Administration Mode: 

Aqe: . 

Packaqe- Exercise: 
National P-value: 



Timinq: (in seconds) 



Multiple choice 

Machine 

Group 

03-29 



RA01 



RA"01 



- Stimulus: 
Response: 
I don It know: 
Pause: 
Exercise total: 



75.21 



10 
7 
5 
6 

.28 



31-4 . 

V-102 



ERIC 



• Selected NAEP Mathertiatics Items 

762 =. 

■ « 

ciD .7 + 6 + 2. 
^ (ZD 7 + 60 + 200. 

700, + 60 + 2. ' 
CD 70 + 60 + .20. 

CD I don't know. 



RH02- 
□BU \l\ 



ERIC 




"^^^ r^«-^a I DO NOT CONTINUE ' 

V-1Q3 • IbTOr I UNTIL TOLD TO DO SO 



ilOM-l 



Report «: 
N4EP f: 

Content &cea: 
Oblective: 



Selected NAEP Mathematics J terns ' • 
RA02 

A110U-1 ' 

A. NONBBB KHD NUMERATION CONCEPTS 



III. Onderstand 
"processes* 



Hathematical ^ Concepts 



and 



Exercise Type: 
Scoring Type: 
Administration Mode: 

Aqe: 

Packaqe-Exercise : 
National P-value: 



^Timinq: (in seconds) 



Multiple choice . 
'Mach}.n.e 
*Gi:oap 

07-10 



RA02 



7a,25 





Stimulus: 


5 




Response: 


21 




I don't knov: ^ 


5 




Pause: 


5 


RJ102 


Exercise total: 


36 









FRir 



31G-; 

V-104 



Selected NAEP Mathematics Items 

ttfe firsOthree odd numbers are 1. 3, and 5. What is the next odd number 
after 5? . 



ANSWER. 



RRBH 



ERJC 



C2DC3P 
CEDCID 
ODCZD 
CDCD 
CD 
CD. 
QD 
CID 

do 

CO 
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DO-NOT CONTINUE 
UNTIL TOLD TO DO SO. 



Selected NAEP Mathematics Uems 



Report #: 
NAEP t« 



RAO a 

f 

A2ioba-i 



Content irea: 
pbiective : 



A, NUMBER AHD H0SBB4TI0N CONCEPTS 

7. Dsing Mathematics and Mathematical Reasoning 
To Analyze ' Problem ^ Situations, Define 
Problems r Formulate ^ Hypothesesr Sake 
Decision^r and Verify Results, 



Exercise Type: • ^ 
Scoring Type: 
Administration Mode: 

Aqe: 

Package-Exerdise: 
National P-value: 



Short answer 
Semi- professional 
Group 

.02-03 



RAOt 



Timinq: .(in seconds) 

Stimulus: 
Response: 

RAOi* Exercise .total: 



80.30 



8 
11 
19 



318 
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Selected NAEP Mathematics Items 



SCORING GUIDE 
RAOU 
A2100a-1 



00' = Ko Besponse 

10.' ^ 7 

11 = 7,9,11 



12 = Gives 7 first plus' othep odd number (s) ; 
e.q., 7,9;"77Tl; 7,9,13' 



20 = 9 

21 = 6; 6,7,.. . (increments by 1 each time) 

22 = "2,4, 6»' 

23 = 3 . 

24 = Other .Unacceptable 
3^ = I Don't Know. 



3ia 

V-107 



ERIC 



. Selected NAEP Mathematics "items 



•Counting by 1.0's, ^vhat n.umbef comes next?" - 



10, 20,. 30,. 



ANSWER. 



RHCSS 



aau I 



CZDCID 
CDdD 
CIDCZD 
ODCX) 



CD ^ 



czD 320 

S ^ V-108 ■ [3JQP I NOT CONTINUE 




UNTIL TOLD TO DO SO. 



!ion-t 



Selected NAEP Mathematics Items , 



Report 
NARP t: • 



R\0.6 

A210.1 1-1 



Content Area: 
Oblective : 



A. HOHBER AND NOHBRATION COHCBPTS 

I, Recall ^nd/6r Recognition of Definitions^ 
PactSr and Syrjbols. 



Exercise Type: 
Scoring Typa: 
Administration Mode: 

Age: 

Package-Exercise: 
National* P-value: 



Timing'*: . (in seconds) 



Short answer - 
Semi-praf essional 
Group 

' i 

' 03-18 



RA06 



HA06 



Stimulus: 
Response: 
Exercise total: 



93,78 



6 
12 
18 




' " Selected NAEP Mathematics Items 



SCOaiHG QUI-DE 
RA06 
\21011-1 



00. = No Response 

10 = 40; 10,20,30,1*0 



*11 = Any response by 10's beginning with. fiO; 
e.q., 40,50 



. 20 = Other Unacceptable 
21 = 20 



39 = I Don't Know. 



. 322 
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Selected NAEP Mathematics Items 



Which one of the following is the sum of three hundreds, eight tens, and four 
ones? 

CD J 5 

mm, 384 

cz) 300,804. 

4 

CD kaon'tknow. > « . 



OBiJ m 




DO NOT CONTINUE . 
UNTIL TOLD Ip DO SO 



SeUcted NAEP Mathematics Items 



Report #: 
NAEP I: 



BUG? 



A2t016*-12 



Contant Area: 
Oblective: 



A. HOBBBR AND NUMEBATIpN^COjlCBPTS 

-III. Onderstand Hathematical Coacepts 
Processes. 



and 



Exercise Type: 
Scoring Type: 
Administration Hode: 

Aqe: 

Package-Exercise: 
National P^.value: 



Timinq: (in seconds) 



Multiple choice 
Machine 
Group . 

► 

_9 _ 

04-33 



RA07 



- Stimulus: 
Response: 
I don't know: 
. Pause: 
Exercise total: 



73.81 



9 
31 
5 
6 
51 



13__ 
09- 1 3 

91. .36 



10 
31 

5 
6 
52 



'324 
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Selected NAEP Mathematics Item's - 
A. What fractional part of the figure below is shaded? 



ANSWER. 



B. What fractional part of the figure below is shaded? 



^RRf 0 
LIBd Hi 




ANSWER. 



EMC 



A 

coco 
coco 

CIDCID 
CSDCSD 
C3D 
CjD 
CD 
CD 

CD 



B 

" CSDCSD 
CIDCID 
' CIDCZD 
CSDCID 
CO 
CO 
CID 

cr> 

CD 
C2D 



321) 



v-n3 




DO NOT COMTINUE 
UNTIL TOLD TO DO SO, 



Selected 'NAEP- Mathematics Items 



'(Continued) " ' , 

C. What fractional p>art of the figurc-'below is shaded? 




ANSWER 



D. What fractional part of the figure below is shaded? 



ANSWER. 



RHIX3 
"DBUTir 



ERIC 



c 




CD CO 


C2DCI3 


CIDCED 


CIDCD 


COCTD 


CDCD 


ODCX) 


CDCD 


C2D 


CD 


CO 


CO 


CD 


CO 


CZD 


CD 


CD 


au 


CD 


CD 
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STOP 



DO ,NOT CONriNUE • 
UNTIL TO LDfe»rO DO SC: 



3004-1 



Report #: 
HABP t: 



Selected NAEP Mathematics Items 

RAIt) 



Content Area: 
Obiective: 



A, NUMBER AN,D NUMERATION CONCEPTS 

III,. Onderstand Bathematical Concepts 
Processes, 



and 



Exercise Type: - 
Scoring Type: 
Administration Mode: 



Multiple part short -answer using drawings 

Semi-professional 

Group 



Aqe: 




9 


Package- 


Exercise:- ' 


02-26 


National 


P-value : 






RA10A' 


30;80 




RA10B 


31.30 




* RA10G 


30.60 




' ' RA10D 


36.^0 


Timing: 


(in seconds) 






Stimulus: 


5 




Response: 


26 


RA10A 


Part total: 


3 V 




Stimulus": 






Response :. 


'26 . 




^ Turn page: 


(6) 


RAiaB 


Part total : 


30** 




• Stimulus: 


3 




Response : 


26 


RAIOC 


' Part total: 


29 




- 'stimulus: 






Response: 


26 


RA10D 


Part total: 


'30 


RAIO 


Exercise total: 

r 


120 



♦♦Time to turn page is not included in total tim^s. 



327 
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Selected NAEP Mathematics Items 



SCORIHG GUIDE 
RA10 . 
A2300l»-1 



Part A. 00 = No Response 

10 = 1/U; .25; 25%; a quarter; 1 oqt of 4 



20 = Attempt to hame shaded part; e.g., second, 
1, rectangle 



' 21 = Other Unacceptable 
22 = 1/3; 3/1; 1-3 

4 • 

39 = I Don't Know, 
Part B, 00 = No Response 



10 = 1/3; ,33; 33%; one-third; 2/6; two-sixths; 
2 out of six 



20 = Attempt to name shaded part; e,q., 2-5^ 1-^r 
triangles 



21 = Other Unacceptable 
2^ = 2/U; U/2; U-2; 2-4 . 
39 = I Don't Know* ^ 



328 
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Selected NAfP Mathematics Items 



V SCORING, GUIDE 
RAID 
A23004-1 



No Respons€i_ 

2/5; .aO; 40%; 2 out .of 5; tvo-fifths 



ittempt to name shaded parts; e.g.^ left 
ones, one side^ trianqles 



Other Unacceptable 
2/3; 3/2; 2-3; 3r2 
I Don't Know. 
No Response 



1/,2; .SO; 5D%; a half^; 2/4; two quarters; 
4/8; 4- puH: of 8 ^ 

< 

Attempt to name shaded parts; e.g., 2 
half-rovsr* corner ones, 1,2,7^8, rectangles 



Other Onacceptable 
4/4; .4-4; 2/2; ,2-2 
I Don*t Know. . 



329. 
■ v-n? 



Selected NAEP Mathematics Items 



A. candy bar is broken into three pieces of the same size. Each piece 
is what part of the candy bar? . 



ANSWER 



OHJ r 



ERIC 



C2DCO 
CDCZD 
CE3C±) 
COQD . 

' CD 

cn 

CO 



330 
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ICT/MkI N^"^ CONTINUE ' 
lalli^rl UNTIL TOLD 10 DQ SO 



1 



Selected NAEP Mathematics Items 



Report #: 
NAEP »: 



BA12 
A230,11-1 



Content irea: &. NUHBBR and NOHEHATION CONCEPTS 
Oblective: ii. Perform Hathrematical Hanipulations^ 



Exercise Type: 
Scorinq Type: 



Short answer 
Semi-professional 



Administration Mode: Group 

Aqe: V 9 

Package- Exercise : 01-02 

National P-value: ' * * 

, ' RA12 ^ 19,90 

Timing: (in seconds) 

Stimulus: 6 

Response: 16 

RA12' Exercise total: 22 



ERLC 
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Selected nXep Mathematics Items 



SCORING aOIDE 
RA12 
i2J011-1 



00 = Ho Response - . 

10 = 1/3; .33; 33«; one third 
'20 = 3; 3 pieces 



21 = .End, middler end; 2 sides ^ tind middle; 2 
sides; , middle; ends; left . 



22 = Other Unacceptable 
39 ^' I Don't Knov/ 



332 
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Selected ..NAEP. Mathematics Items 
A. Which number is GREATER? 

mm 3,000,000 , 
cz) 800;000 



CZ) I don't know. ^ 



B. ^ Which number is GREATER? 



<=D 3.000 
mm 3,200 

CD Ldon't know. 



□BJ t 



V-]2] 




STOP 



DO NOT CONTINUE 
UNTIL TOLD TO DO SO. 



Report t: 
NAEP f: 

Content Area; 
Obiective: 



Selected NAEP Mathematics Items 
RB02 ^ ' 



B11010-12 • 



pbopertir:^ op nuhbees and operations 

I. Recall and/or Recognition of Definitions < 
Pacts, ,and Symbols. 



Exercise Type: 
Scorinq Type: 
Administration ' Mqde: 



7 



Multiple part multiple choice 

Machine 

Group 



Aqe: 






9 


. _13__ 


Packager Erercise : 




02-24 


0.8-30 


National 


P-Talae: 


> 




( 






RB02A 


81 .93 


95,92 






. 8B02B 


85.84 


"96.80 


Timing: 


(in seconds) 












Stimulus: 


6 


6 






Response: 


^6 


16 




I 


donH know: 


5 


5 






Pause: 


6 


6 


RB02A 




Part total :' 


' 33 


• 3.3 . 






Stimulus: 


6 


6 . 






Response: 


16~ 


17 . 




I 


don't know: 


5, * 


5 






Pause: 


6 


6 


" .RB02B 




Part total: 


33 


34 . 


RB02 


" Exercise total: 


66 


67 " 
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Selected NAEP Mathtematics Items 
Do each of the problems below. . . 

A. 3 + 0 a 



A. ANSWER, 



B. 3 X 0 « 



B. ANSWER. 



C. 3 - 0 = 



C. ANSWER. 



OBJ II 



FRir 



A 


B 


c 


CSDCSD 


CSDCSD 


CSD CD 


, CDdD 


GDtED 




dDCO 


dDCID • 


CtDOD 


dDCD 


CSDCjD 


CJDCJD 


CD 


CSD 


C2D 


cx> 


OD 


.CD 


CSD 


CSD 


CJD 


CD 


.CO 


CED 


CD 


CSD 


CD 


OD 


CSD 


CD 
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STOP 



DO NOT CONTINUE 
UNTIL TOLD TO DO SO. 

■ ;t?' • 



Report i: 
HASP I: 



S'elected /lAEP Mathematics Items- 
RB03 

B12005-li 



Content Area: B. PRO^EHTIES OP NOMBERS iND OPERATIONS 
Obiective: li. Per/fom .Hathematical Hanipulations. 



Bxetcise Type? 
Scoring Type: 
Administration' Hodej 

Aqe: . • 

l^acJcaqe- Exercise : 
National P-ralue: ' 



Baltiple part short answer 

Semi-professional 

Group 



RB03A 



RBQ3B 



t»B03C 
RB03. 



9 

06-32 



^3 _ 

05-oa 



RB03A 


94.26 


98. 


• HB03B 


81 .49 


9a. 


RB03C 


87.76 


94. 


ihds) 






•Introduction: 


3 


3 


Stimulos: 


3 


3 


Response: 


16 


15 


Part total: 


. 19 


' la 


Stimulus: 


3 


• 3 


Response: 


17 


15 


Part total: 


20 


18 


Stimulus: 


4 


3 


Response: ' 


16 


1'6 


Part total: 


20 ■ 


19 


Exercise total: 


62 


58 



ERIC 
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Selected NAEP Mathematics Items 



SCORIHG GOIDB 
RB03 
■ B12O05-12 



Part A.- 00 = No Response 



10 = 3 



20 = ,0 



21 = Other Onaoceptable 



39 = 



Part B. OO = 



I Don' t Know , 

Ho Response 



10 = 0 



20 = 3 



21 = Other Onaccep'tabre 



39 = I Dbn't Know. 



Part C. OO = Ho Response 



10 



20 = 0 



21 = Other Unacceptable 



39 = 1 Don't Know, 



ERIC 
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Selected NAEP Mathematics It'ems 

Which one of the following is equal to ,3 X 5? 

CD 3 t 3 + . 3 

cbS + 5+ 4 + S + 5 

• 3 + 3 + 3 + 3 + 3 

CD3 + 5 + 3 + 5 + 3 + 5 

4 

. CD I don't know. 



RBtSH 



/V 



ERIC 
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CT^NO I NOT CONTINUE 
^Iv/r I UNTIL TOLD TO DO SO. 



Selected NAEP Mathematics Items 

Report #: jb04 

NABP #:" B12008-1 

Content Area; PHOPEHTIBS OP NOMBBHS iHD OPEHATIOHS 

Obiectiire; ll. Perform Hathematical Hanipulations. 

Exercise Type: Multiple choice ^ ^ 

Scoring Type; -Machine 
Administration Mode: Gro;ap S 

Age: 9 

Package- Exercise : 05^2 8 

Nati'dnal P- value; 



RB04 73.29 



Timing: (in seconds) 



Stimulus: 7 

^ Response: 31 

I don't knov: 5 

Pause: 6 

RBOa Exercise total: a9 



339 
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Selected. NAEP- Mathematics Itenis 



315 
179 

136. 



What two numbers could you add to check this subtraction? 



ANSWER 



and. 



□BU ii 



ERIC 



COC2D 
dDCD 
COC2D 
CSDCJO 
OD 
CO 
C3D 
CD 
OD 



340 
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ev^A I DO NOT CONTINUE 
5TOr I UNTIL TOLD TO DO SO 



Ill'Bl 3001 -i ,2.3,4 



Report #: 
NAEP •: 



Selected NAEP Mathematics Items 
RB06 

Bt3b01-123a 



Content Area: 
Objective: 



B. PHOPBHTIBS OP HOMBEaS AHD OPBRATIOMS 
•II* Perform Hathematical Barnipulations* 



Exercise Type: 
Scoring Type: 
Administration Mode: 



Short ansver 

Semi-professional 

Group 



Aqe: ' ' • 




9 


13 


17 


idult 


Packaqe- Exercise : 




05-33 


02-02 


09-11 


02-02 


National P-value: 














RB06 


ai,90 


82.17 


88.^2 


86.29 


< 

Timinq: (in seconds) 














Stiiiialus: 


11 


U 


10 


11 




Response: 


19 


21 


20 


* 


RB06 Exercise total: 


30 


32 


30 


* 



♦Time not limited by paced tape. 



34i 
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Selected NAEP Mathematics Items 



10 = 



SCORING GUIDE 
RB06 
. >B13001t123U 



>< X- 



00 = ^Ho Response — ^ 



179 aad 136; 136 and 179; 136 + 179; 
179 ♦ 136 . ' - . 



11 = Has 139 or 163 for 136 plus 179; has .176 or ' 
'197 for 179 plus 136 , ' , ' . 



20 = 315 + 136j 315 ♦ 179 

' • ■ ( \- . ' 

21 Other ynacceptable' 
39 = I Don't Know. 



Note: Only Category ia wa^^^ considered 
acceptable. \/ 7'^ 



342 
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Which fraction is the GREATEST? 



CD J 



4 
4 



3 

.8 



CD I don't know. 



OBJ IV 



343 

.V-131 



ERIC 




DO NOT CONTINUE 
UNTIL TOLD TO DO Sf 



Selected' NAEP Mathemat'lcs Items 



Report #: 
HASP I: 



RB10 

B22017-123 



Content Area: 
.Obiective: 



PROPERTIES OP HOSBEHS. AND OPERATIONS 

IV. SolTinq Hathematical Problems — Sociair 
Technical r and Acaaemic. " ' . 



Exercise Type: 
Scoring Type: 
Administration Hode: 

Aqe: 

Package-Exercise: 
National P-value: 



Timing*: {in seconds) 



Multiple choice 

Machine 

Group 



RB10 



06-21 
3.22 



-13 « 
02-27 

26. 18 



17 ^ 
04-12 



49.16 



RB10 



Stimulas: 


7 


8 


8 


Response :. 


119 


99 


99 


I don't know: 


5 


5 


5 


Pause: 




6 


6 


Exercise total: 


137 


118 


1 18 



ERIC 



341 



Selected NAEP Mathematics Items 
Add the following numbers: 

$ 3.06 ' . - 

10.00 ^ . 

9.14 

. . 5.10- . i> 



ANSWER. 



r. » 



RC0I 

□5 J . il 



345 



CSDCSD 
GDCZD 

CD CD 
CD 
CD 

CZD V'-133 fer^nl NOT CONIINUE 

CZD - I STOP I UNTIL TOLD TO DO SO. 

CD 
CD 




Report #: 
HABP »: 

Content Area: 
Obie<il:ive: 



-^^ Selected NA£P Mathemtics Items 
RC01 

C10011-123U . 



V 



C. IRITHIETIC COHPOTATION 
II. Perform Hathematical Sanipulations. 



Exercise .Type: 
Scorirtq Type:- 
Administration Mode: 

Hqe: 

Packaq^e- Bxerci se,: 
National P-Tralue: - 



Timing: (in aeconds) 



Short ansver 
Seihi- professional 
Group 



Rcor 



RC01 



stimulus: 
Response; 
Exercise total; 



_9 t 
06-04 



♦Time notr limited 'by paced tape. 



1 

42 



13 



1 

31 
32 



02-0 1 



39.85 84.34 '^2.45 



1 

31 . 
32 



A doit 
O6-0I 

86. 18 



1 

4c 



ERIC 



34 G 

'V-134 



ERIC 



Selected NAEP Mathematics* Items 



SCORING GUIDE 
RC01 
C10011-1234 



00 = No Response 



10 - 127.30 

C if' 

11 = 27.30 



20 = 2730; $2730 {any decimal error) 

21 = -27.20; $27.20; 17.30; $17.30 (can misplace 

decimal) 

22 = 17.20; $17.20 (can misplace decimal) 



23 = 117219; $117210 (can misplace decimal); 
11721 ' 



2,a = Other Unacceptable 

-39 — F — rTrTKnT»-t~irn:o - " 



^ 347 

V-135 



Selected NAEP Mathematics Items 



Do the problems on this and the following page. 



A; Add: 



38 
+ 19 



ANSWER 



B. Subtract: 



36 
-19 



□au il 



ANSWER 



ERIC 





B 


CSDCSD 




CZDCID 


GDCED 


CEDCXD 


ODCO 


CDC3D 


CDC3D 


CSD 


OD 


CID 


CD 


(3D 


CO 


' CD 


cn> 


OD 


CD 


CD 


CD 



318 

V-136 




er^B I NOT CONTINUE 

OTOr I UNTIL TOLD TO DO SO. 



Selected NAEP Mathemtids Items 
(Qontinucd) 
C. Multiply: 



38 
X 9.. 



ANSWER 



D. Divide: 



5 \l25 



ANSWER 



RCB2 



□Sut 




c 


D 


C5DCO 


' C2DCD 


.COCO 


CD CO 


COGD 


COCZD 


CDCD 


CIDCO 


CO 


• CO 


CO 


.dD 


OD 


CO 


CO 


. CO 


" CO 


CO 


OD 


CD 
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iev^Bf NOT CONTINUE 
loTv^rl UN*TIL TOLD TO DO SO 



9017-1. 



Report f: 
NiEP #: 



Selected NA£P Mathematics Items ?• 
RC02 

Cr0017-1234 



Content Area: 
pbiectiTe: 



C. ARITHMETIC COBPOTATION 
II* Perform Hathematical Manipulations* 



Exercise Type: 


Multiple 


part short 


answer 






Scoring Type: 


Semi-professional 








Jldministratioa Mode: 


^r-aap ' — 










Aqe,: 




9 


13 


17 


idult 


.i?acitaqe-Bxercise: 




01-05 


06-13 


07-09 


0*1-14 


National P-valae: 














RCJ02A 


79.00 ' 


94.30 


96.76 


96.9^1 




RC02B 


55.03 


88. 90 


. 91.58 


91.48 




RC02C 


25.23 


82.56 


87. U7 


80.75 




-RC02D 


15.23 


88.53 


93.17 


92.77 



Timinq: (in seconds) 





Introduction : 


4^ . 


4 


4 




Stimulus : 


2 


2 


; 1 




Response : 


30 


30 


30 


RC02A 


Part total: 


32 


32 


31 




Stimulus: 


2 


2 


2 


V 


Response: 


30 


31 


31 




Turn page: 


(6) 


(6) 


(6) 


RC02B 


Part total: 


32** 


33** 


33** 




Stimulus : 


2 


1 


1 




Response: 


30 


31 


31 


RC02C 


Part total : 


32 


32 


32 




Stimulus : 


1 


2 


1 




Response: 


31 


31 


31 


RC02D 


Part total: 


32 


33 


3? 


RC02 


Bxercise total: 


132 


134 


132 



♦Time not limited by paced tape. 
♦*Time to turn page is not included do total times. 



4 
1 



350 

V-138 



ERIC 



. Selected NAEP Matheniatics' Items 



SCORING GOIDB 
RC0 2 
C10017-1234 



ParJt A. 00 = No Response 

10.. _5_7 



20 = «17 

21 = UT 

22 = 19 1 29 

?3 Other Onacceptable 
39 Don't Know. 

Part B. 00 = No Response 

m .= 17 

20 = 27 . 

21 = 55 
22' = US 

23 = Other Onacceptable 

24 = 23 

* 

39 = I Don't Know. 



ERIC 



Selected NAEP Mathematics Items 



SCORING GDIDE 
RC02 
C10017-1234 



PatTt C. 00 ' = No Response 



20 - 272 



21 = 2772 



22 = 297 



23 = Other anacceptable; e.qi, 972, 432 



39 = I Don't tnov. 



Part D. 00 = No Response 



10 = 25 



21 = Other Onacceptable 



39 = I Don*t Know, 



352 

V-140' 



ERIC 



Selected' NAEP Mathematics Items 



Do the following subtraction: 



1,054 
- 865 



V 



ANSWER. 



/ 



RCBH 



□BJ il 



) 



ERIC 



COCD 
C2DC!D 
C3DC3D 
CXDOD 
CO 
CD 
" CO 
CD 
* CD 
CO 



,353 

v-141 




I CT^B I '^'OT CONTINUE 
I O lV^r J UNTIL TOLD TO DO SO 



Selected NAEP Mathematics Items 



Report I : 
HAEP t: 



Cl0021r1234 



Content Area: 
Obiective: 



C. ARITHMETIC COHPUTATION 
II. Perform Mathematical Manipulations. 



Exercise Type: 
Scorinq Type: 
- »^ini n i S t r a t i on-^hyie^- 

Aqe: 

•Package-Exercise: 
National P-value: 



Short answer 
Semi-professional 

-ijr onp 

-2 _ 

02-06 



RC04 



Timing: (in seconds) 

Stimulus: . 
Response: 

RCOt Exercise total: 

*Time not limited by paced tape. 



27,17 



2 
40 
42 



13_ 
02-31 

80, 02 



1 
40 
41 



_12_ 

08-09 
88.84 



1 

41 

42 



Adult 
0?-17 



89.71 



2 



V-142 



ERIC 



Selected NAEP Mathematics Items . ' _ . ' 

* SCORING GUI'DE. . ' . 

RCOU " . 

Cia021-123U 

, t . f 

s 

r 

OQ = No Response 
10 = 189 

20 = ' 299 

21 = 199; 289 

22 = Any attempt to add; e.q., 1919; 18119; 18 

23 = Other Unacceptable 

24 = 211; -211; 1811; -1811; 11; -11; 8-11; -81 

25 = 1189; 1299; 1199; 1289 • 
39 = ^ I Don' t Know* . 



O i) O 
V-143 



Selected- NAEP- Mathematics Items 



Betty's dog eat« two "biscuits every, day. How many d-ays wlU it. take the dog 

• ' , 'I ' , ' ■ ' •' 

to finish a package of24.biscuits? '■ ' . 



ANSW.ER. 



RC07 



□BJ IV 



ERLC 

5'-CI0O3tt. 



CD CD 

GDCSD 
CO 
CD 
CD 
CD 
CD 
CD 



V-144 




STOP 



DO NOT CONTINUE 
UNTIL TOLD TO DO SO. 



Selected NAEP Mathematics Items 



Report #: 

^HAEP #: 
/ 

Content Area: 
Ob^iective: . 



RC07 

C10036-1 

C. ARITHHBTIC COMPOTATION ' . 

IV. Solving • Mathematical 
Technical , and Academic • 



ProbI.ems--SociaIr 



Exercise Type: 
Scoring Type:'. 
Administration Mode: 

Age: / ' \ 

Pac k aq e- Ex e r ci s e :. 

Nationjil PTyalue:, 



Short answer 

Semi-professional 

Group 

_9 

. 05-01 



RC07 



Tinjijiij:" (iri seconds) 

. ■ ' . . Stimulus: 

• » ^. , ^ Response: 

'rc0^7 Exercise total: 



36.71 



8' 
30 
38 



V-145 



ERIC 



.Selected NA^P Mathema'tics Items 



SCOHIHG GOIDE 
' ' BC07 
C10036-1 



00 = Ho Response 

« 

10 = 12; 12 days - ^ . . 

11' = "12 with wronq units; e.g., 12 biscuits 

12 = 2i»/2; attempt to divide 24 by 2 

20 = 22 (days) ; attempt to subtract 2 from 2H 

21 . = 26 (days): attempt to add 24 and 2 

22 = 48 (days) ; attempt to multiply 24 by 2 

23 = 24 . 

24 = Other Onacceptable 
39 = . I Don't Know, 



Hote: Only Categories nO and 11 were 
' considered acceptable. 



358 

V-146 

ERIC . - . 



Selected NAEP Mathematics Items 
10 X 10 X 10 X 10 = 



ANSWER 



aBd U 



er|c • 



CO CD 
CTDGD 
CZDOD 
CJDCD 
CD 
CD 
' CD 
CD 
CD 
CD 



V-147 




STOP 



DO NOT CONTINUn 
UNTIL TOLD TO DO SO 



S-Cld057-I. 2.3 



Selected NAEP Mathematics Items 



Report #: 
NAEP »:. 



BC09 

C10057-123 



Content- Area:/ C. ABITHNETIC COHPOTATIOH 

Obfective^ II. Perform Hathematical Manipulations. 



Exercise Type: 
Scorinq TTpe: • 
Administration Hode: 

Aqe: 

Package-Exercise: 
National P-value: 



Timinq: (in seconds) 



Short answer 
Semi-professional 
Group « 



1_ 
06-25 

a.27 



RC09 



RC09 



• Stimulus: 
Response: 
Exercise total: 



26 
30 



^3 _ 
03-30 

67.07 



-4 
27 
31 



V 
11-27 

79.71 



H 
26 
30 



ERIC 



3 GO 

V-148 



Selected NAEP Mathematics Items 



SCORING GDIDE 
' RC09 
C10057-123 



r 

* 00 ' = . No Response 

10 = 10,000; 10000 

11 = 10 to the 4th power 

20 = 40; 400; 4000; 40,000' 

21 = 200 

22 • = 10... where the number of -0«s is not 

23 = Other Onacceptable 
39 = I Don't Knotf. 



361 

V-149 



Selected NAEP Mathematics Items 



Report #: 
HiEP »: 



RC1U 

C20008-1 



Content Area: 



Obiective: 



C. AEITHMETIC COHPOTATION 



III. Understand 
Processes. 



Mathematical 



Exercise Type: 
Scorinq Type: 
Jdministration Hode: 

Age: 

Packaqe-Exercise: 
National P-ralue: 



Timinq : (in seconds) 



^hort answer usinggr^ 
Semi-professional 
(iroup 

05-23 



RC1U 



RCIU 



Stimulus: ^ 
Response: • 
Exercise total: 



21.37 



' 6 
'30 
36 



V-150 



4 ■ 

Selected NAEP Mathematics Items 

SCORING GOIDB 
^ RC1U 

C20008-1 

00 = So Response 

10 6; 6 dots; 6 circles ' 

11 = 3/4; 3/4 of .the dots {or circles) 

20 = 2; 2 dots (or circles) 

21 ' = t»; «» dots (oir circles) ; 1/2; 1/2 of the dots 

(or circles) 

22 = 1/4; 1/U of the dots (or. circles) 

23 = Other Unacceptable 
.39 = I Don't Know. 

> 



3G3 

V-151 



1 



-Selected NAEP Mathematics Items 



Do the following multiplication: 



2^4 



ANSWER. 



RCIB 



OBU II 



CZDCO , 
CZDC3D 
CE)Gr> 
CED 
C3D 
CZD 
CD 

ao 

CO 



3G4 



V-152 




STOP 



DO NOT CONTINUE 
UNTIL TOLD TO DO SO. 



Selected NAEP Mathematics Items 



Report- #: 
HASP t: 



HC16 

C20022-23 . 



Content Area: 
Obiectire: 



C. ARITHMETIC COBPOTATION 
II. Perform Hathematical Hanipolations. 



Exercise Type: 
Scoring Type: 
Administration Mode*: 

Aqe: 

Packaqe-fixercise: 
National P -value: 



Timing: (in seconds) 



Short answer 
Semi-professional 
Group , 



EC16 



RC16 



Stimulus: 
Response : 
Exercise total: 



_13_ 
05-26 

62. 25 



2 
20 
22 



17 

09-02 

73.51 



2 
19 
21 



365 

V-153 



ERIC 



Selected NAEP Mathematics Items 



SCORING 60IDE 
RC16 
C20022-23 . 



V 



00 = Hp Response . 

10. = 1/8 

20 = 2/8; 1/4 

21 = 3/H 

22 = 8 

23 = 2/H; 1/2; 4/8 

24 = Other Unacceptable 

25 = 1/6 

39 = I Don't Know. 



ERIC 



3CG 

V-154 



Selected NAEP Mathematics Items 

John has 382 stamps in his stamp collection, Gfeg has 224, Pete has" 310 and 
Bob has 175. The number of stamps the boys have altogether is CLOSEST to 
which one of the following numbers? 

CD 900 

CD 1000 

mm 1100 . , / 

CD 1200 • . . ' / 



CD I don't know. 



./ 



RC21 
OBJ IV 



-.367 

V-155 




eTr\D I ^0 NOT CONTINUE 
OiWrl UNTIL TOLD TO DO SO. 



Selected NAEP Mathematics Items 



Report #: 
NAEP I: 



RC»1 

C60001-1 



Content Area: 
Oblective: 



C. ARITHflETTC COnPOTATION 

IV. Solvinq ' Hathematical , 
Technical^ and Academic, 



Problems — Social , 



, Exercise TS^pe: 
Scoring Tyi 
Administrajdoh Mode: 

Aqe:w 

Packaqe-Erercise : 
National P-value: 



Timinq: (in seconds) 



Multiple choice 

Machine 

Group 

9 i 
05-30 



RC21 



RC21 



Stimulus: 
Response : 
I don't -knov: 
Paus;e: 
Exercise total: 



30.57 



28 
31 
5 
6 
70 



ERIC 



.368 
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Selected NAEP Mathematics Items 

A person is standing on the scale below weighing himself. How much does he 
weigh? . ' ' ' • 




CZD 59 pounds 
pounds . 
62 pounds 
cny^ 63 pounds 

I donH know. 




RE:.12 
□Bd II 




DO NOT CONTIHUE 
UNTIL TOLD TO DO SO. 



ERIC 



' Selected NAEP Mathematics Items 



Report #: 
NAEP ♦: ' 



RE12 

Et4012-12 



Content Area: 
[ ObiectiVe: 



E« ESTIHATION AHD tfEASO^BHEMT 

Perform Mathematical tfanipulations, 



Exercise *Type: 
Scoring Type: 
Administration Node: 

Aqe: - ' . 
Packaqe- Exercise: 
National P-value: 



(in/seconds) 




Multiple choiqe using graphs table, or drajiing 
Machine * * - 



Group 




RE12 



RE12 



„9 _ 
02-33 

80,03 



13 
04-16 

91.02 



stimulus: 


15 


14 


Response: 


15 


16 


I don't knov: 


5 


5 


Pause : 


6 


^ 6 


Exercise total: 




41 





ERIC 



370 
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Selected NAEP Mathematics Items 



3 + □ = '10 



Replace the box with a numb.cr to make the above statement TRUE. 



ANSWER. 




RHElV 



ERIC 



CDCXD 

ay 

CSD 
C3b' 

,. CZD 
dD 

' '.'cD 



371 

* V-159 




STOP 



DO \(yi- ( ON IINUr. 

UN I II. TOI.D • l() 00 SO 



S'tlttOOt-l 



Selected NAEP Mathematics Items 
Report / , SHOI * 

H&BP t: H1 1001-1 



. Content Area: H. EQUATIONS ASD INEQUALITIES 

^' \ ■ ' • ■ 

Obiective: III. Opderstandt flathematical 

/ _ t Processes, 



Exercise' Type: Short answer - 

Scorinq -^ype: Semi-professional 

Administ^a^tijon Mode: . Group 

Aqe: ' 9 

Package- Erercise: 03-21 
National P-value: 

. RH01 89.88 

timing: (in seconds) 



Stimulus: • • 4 
Response: 26 
RH01 Exercise total:/ 30 



:372 
v-ieo • 



ERIC 



0' 



Selected NAEP Mathematics Items 



' SC08JNG GUIDE 
RH01 
H110Q1-1 



00 =1 No Response 
10 = 7 



11 = Correct' process vitJi 'no ariswer or wrong 
' answer*; e.g., 10 - J 



20 M3; any atj^pt to add 

21 = Qtber Onacceptable; e»g*, 17- 

3$ = I Doa't' Know. ' , " ' ^ ^ 

Bote: Only. Category*- 10 was consideced 
acceptatiie. , " 



4 



373 

V-161 



Selected Ni^EP Mathematics Items 



Wluch.onq of the follow.ing figures is a rectangle? 



cz> 




' I don't'know. 



MBd I 



374 

V-162 , 




.OlWrl i.Mii.ioi.I) TO SO. 



Selected NAEP Mathematics Items 



Report #; 
HABP t; 



RKOS 
K20O01-1 



Content Urea: 
Oblective: ^ 



K. GEOBETRY 



I. Recall and/or Recognition of Def initionsr 
Pacts, and Symbols. 



Exercise Type: 
Scorinq Type: 
Administration Bode: 

h-qez- ' % 

Packaqe-Exercise : 
National P-valuet 



Timing: (in seconds) 



Bttltiplej choice using graph, table, or drawing 
Machine - 
Group. 



PK05 



Stimulus: 
Response: 
I d6n!,t knov^: 
Pause: 
•Exercise total: 



9 

03-T» 

7U.03 



6 
20 
5 
6 
'37 



Selected NAEP Mathemattcs Items 




What is the figure above called? 



RKB5 



ANSWER. 



□BJ I 



00 



ERIC 



CD CO 

CSDOD 
C3DCD. 

CD 
CD 
CD 
, CD 
CD 
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STOP 



DO NOT CONTINUE 
UNTIL TOLD TO DO SO. 



Selected NAEP Mathematics Items 



Report »: 
HAEP i: 



RK09 

K20041-1 



Content Area: 
Obiective: ' 



K. GEOMBTBY 

!• Recall ajid/or: Recognition of Definitions, 
Pacts, and Symbols* 



Exercise Type: ^ 
Scoring Type: 
Administration node: 

iqe: 

Package -Exercise: 
National P-\ralue: 



Timing: (in seconds^ 
RK09 



Short answer using graph, table or dr.aiwing 

Semi-professional 

Group 

04-06 



RK09 



* Stimulus: 
Response: 
o Exercise total: 



7K60 



2 
15 
17 



Selected NAEP Mathematics Items 



SCORING GUIDE 
RK09 ' 
K2O041-1 



00 = ' No Response ^ 

10 = Triangle' 

20 ' = Square^ rectangle 

. { 

2.1 .= Other Onacceptable; e.g.,, box 

3i =^ I Don't Knov. 




y 



.C". NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS DATA SUMMARY 
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D. NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS (NAEP)'SIZE 
AND TYPE OF COMMUNITY REPORTING CATE&0J11ES__^__^ 

The NAEp'' categorization scheme is described as: 
Size and Type of Community (STOC) 



The seven size and type of cornfnunity .(STOC), reporting 
categories are comprised of three "extreme"' 'types of^ 
community (TOC) and four "residual" sizes of community 
(SOC). Each TOC category includes approximately 10% 
of ^he respondents at each age level; the remaining 
respondents are classified according to one 6f the 
SOC classifications. 

Briefly* the three TOC categories are (Dxity areas 
where a iiigh proportion of the adult population is 
either not regularly employed or on wlefare and a low ^ 
proportion is employed in professional or mana^eri^l 
positions; (2) rural arreas where a high proportion of 
adults are farm workers and a low proportion are pro- 
fessional, managerial or factory workers; and (3) 
near-city and city areas where a high proportion of 
adults are employed in professional or managerial 
positions and a low proportion are^factory or farm 
workers, not regularly employed or on welfare. 
Respondents are placed In one of these categories if 
the occupational profile and location of the school If 
or, in the case of the out-of-school sample » segment 
satisfy the extreme TOC definitions. 

The remaining respondents at each age level are 
classified according to the size of community in 
which the school or segment is located. The occup- 
ational profile is based on the employment categories 
summarized in Exhibit A-3. 

For the ,in-school sample at each age and the supple- 
mentary sample at age 17, 'the school principal of 
each selected school provided estimates to the 
percentage of, students whose parents fit into each , ^ 
occupational category. 



National Assessment of Educational Progress, General Information 
Yearbook. Report No> 03/04-GlY, December, 1974» pp 41-43. 




381 

V-169 



Other occupational data for the out-of-school 
sampel was obtained from census, data in Year 03 
and from ^the respondents themselves in Year 04. 
The definition's used to classify respondents hj 
STOC are presented in Exhibit The occup- 

ational index is computed using the occupational 
categories summarized in Exhibit A-3. 



EXHIBIT Occupational Categories 
.Categories Code 
Professional or Managed al 



Personnel . - 'A 



Sales, Clerical , Technical 
or Skilled Workers 



B 



, Factory or Other Blue 
Collar Workers 



C 



Farm Workers 



D 



Not Regularly Employed 



On Welfare 



F 
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EXHIBIT A-4. National Assessment Sire -and Type 

of Commupity (STOC) Reporting Categories 



Reporting Category Occupational Index*; Description 



Low metro 



E+F-A 



E)i:treme rural * 



' Medium city 



E-(C+2A) 



Main big cit^ 



Urban ^f'rfnge 



High metro 



A-(C+D+E+F) 



\ 



Sample schools or segments 
/in a city or metropolitan 
area of a city with a pop- 
ulation greater than 150,000 
and in the 90-99th' percentile 
of the low metro index 

Sample scHools or segments in 
community with a population 
less, than.,25,000 'and not 
. classified as extreme rural 

Sample schools or segments 
In a city with a population 
between 25>00O and 200,000 
and not classified^gs low 
metro .or high metro^ 

Sample schools or segments' 
within the city limits of a 
city^with a population greater 
.than 200,000 and nat class- 
ified as high metro or low 
metro ' ' 

Sample schools or segments 
Mji the metropolitan are^ of 
a big city but outside the 
city limits and not' class- 
ified as lowmetrcjr, extreme 
rural or high metro 

Sample schools or segments 
in a^city or metropolitan 
area of a city with' a 
population greater than - 
150,000 alid in the 90-99th^ 
percentile on "^he high metro 
i ndex 



*'See Exhibit A-3 . 



E. .STATEMENT BY W. JAMES POPHAM TO THE HOUSE' QENERAL, SUBCOMMITTEE 
ON -EDUCATION ON CRITERION-REFERENCED TESTS 
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Ton can't measure ifilleaKe a tablespoon. But everyone knows that^so 
one tries to. After all. tablespoons were designed to serre a cleai;Jy identifl- 
able weii^urfment function, thus they are never employed for assessing such 
things AS dlsf-ance, sound and heat. Si^ificant problems arise, hgu-evcr. wbea 
we inlssioD of aPmen^rinR jn^trurnent is not so patently obvious, hence it can 
De alstakenly used in situations wheret>y<lt yields apparently- respectable buf 
ixusleading aata. ' ' • " ' 

For there arc seductive dangers associated Tvlth the po5;session of data. VTt 
live l^an Increasingly evIdencc-coDscious society, and the person who can trot 
forlh a sufficiently Impj-essive nrray of data often becomes the wiaaer in 
policy djspmes. .\fter nil. our data-devotee wi4l claim that he has-thc facts ahd 
We other side operates only on intuition. But, quite obviously, tjic qualitv of a 
Cata4>ased ^arprnuient decision depends on the quality of the data. InjudL 
Clous selection of measnrin? instrumeDts is likely to yield Indefensible data 

^^^^ °^ education we arc currently sbfreriac from, the' 
•mmjctlons of a markedly misapplied measurement tradit^OD. 
• •k?'^^.^^*-^ ^'^^'^ respect to the particular bill currently- under consideration- br 
inis CpmmiUee. but because misperqeptions rewrchag appropfrnte inensurement 
•iratcgles may impmse upon one^s nppralsal of comparable legislation it is 
aecc^ry to draw dislljictions .between two major^ineasurement uftthodologies 
'^^^^^f'"^^"'"^ ^^^^ ^^^^^ academic capabintifcs "bf the -nlitioD's 
youth. More specifically, differences wiU be identiiiecj l»etween.ji nomurcUr- 
cnocd mcasurantnS approach and a ^ criicrion^rcjfcrcnccd rnca^urcnimt 
tpproaeh. The purpose of these two assessment strategies will b6 "examined 
tionp with Illustrations of how, if the wrong type of api?roach Is ^utilized, mis- 
leaaing Uata will result. 

mC BABIO DI8n>VCTl05 . ' ^ 

Norro-rcferenrcd nncnMires ore u<>ed to nscertnin nti Individual's perforrtinnce/ 
In relationship to the i»crforipnnce of other Jndlvldunls on the same measuring' 
oerlce. The mc-nnlngfiilne*:^ of n» lndlvl<lual score emerges -.from the comp'^rl- 
ion. It Is bt'Cause Ihe ImlivlUual Is compared >vith some nornia-tive group that 
•uch measures are dt'^c^lbc^^as iiorm^efcrciiced. Most btandardizcd tests of 
achievement or intellectual ability used In this .country can be clnssilird as 
norm 'reference) inrasures. Such tests are desiijn^ed to yield a series of relative 
IHJrformance descriptions, that is. relative to the norm group. It Is' expected 
that we win he nhle to disiinguisli between Mary who ^cor^s at the C5ih per- 
centile (of the norm group) and Uarrj who- scores at the 48tb percentile (of 
the.norm group). 

Crltcrlon^referenced mensiirea nre u.«!ed to ascertain an Individual's, status 
with respect to some c*^*erlon, that K nn explicitly described tvpe. oJC learner 
competenre. It is because the individual's performance Is compared with an 
eilabJi^hcd criterion^ rather than^"the pcrformniice of other indiridnals. that 
these measures are described as criterlon-referenced. The meqninrrfulness of an 
Individual score is not dependent on coinparisons with other Individuals who 
took the test. We n-ant to know what an 'lndi\ idual can do. not how he .stands 
In comparison to others. For example, the dog owrier who warits to keep his 
dog In the back yard ma-j give .the dog a fence.jumping test. The owner wants 
to Ond out how high the dog cnn jump so that the o\^jier cnn build a fence 
hl£h enough lo keep the ddp: In the yard. How the dog compares with other 
doga ia irrelerant Another example of a criterion referenced test would be the 



Bed Croas Seplor LIfesarIng Test, where an Indlridual must display certaJn 
•wlmmlng ekills to pass the examination Irrespective of how well others- per- 
Jonn OD the test. Merely because a g^ou^ of wealt swimmers sigii up. to take 
Ine iifesavlns test on a givfn occasion would not cSean that the best perform- 
ance of that group would oecessarily be hiRh enough, to pass the test. 

BlDfrc-norm-referenced measures aro devised facilitate comparisons among ' • • 

Inaivlduals. It Is not surprlbing that (heir primary purpose Is to make* deci- 
•Ions about Individuals. Whieh pupils should be' counseled to pursue higfier. 
cdjcallOD? wWbh pupils should be advised to attain vocational skills? 'These 
are the kinds of questions one seeks'to answer through the Qse ^)f norjn-refer. 
enced mcasure.v for mpoy decisions regarding ah Individual can best be made 
ny knowDing piore about the '•competition/' Uiat is» .by knowing how other, 
comparable Individuals perform. 

I f;*^*^^P^'^f»^'"'o""r^^^«'renced tests are nlso used to make decisions nbout 
Individuals, there Is usually a difTerence In the context In which such decisions ' ' 

are made. Generally, a norm-referenced measure is employed where a decree of 
f^i^^^^^y-^* required by the situation. Kor example, when there are only lim- 
ited opiinlngs In a company's executive training program, the company is nnx- 
lot)! to identify the best potential tralne{?3. It Is critical in such situations, *• V-173 
IhcreXore, that the measure permit relative comparisons among individuals. On > 
the other. hand, In situations^ where one is only interested In whether nn Indi- 
tidoal possesses a particular competence, nnd there arc no constraints regard- o fVt- 

ing how mftuy Individuals can possess that skill, criterion-referenced measures • OOt) * 

aw preferable. In this ^eosc, criterion-referenced measured may be considered 
ooiolufoJndicators.^ 



Statement by Pophani'(2)' . ^ * 

TUD UT&Arrtin) UEJiap^iicTr momoir 

Fortottjy years !d oor nation woiiare rcllcU licnrHy on the ti'ie of narm-ref- 
ercoced jncanurcr Almost without ezcrptlou. the many (Standardized Achieve* 
.txjent* tcsti "u5wJ ihrouchout the land lit the einsslc •notm-rcfercncc<l rpcasiirc-' 
- tncDt model When these devices wore used iu n fashion coiJsi-ient. wjtU theJr 
ChleT mt.*f9ton, that in,* to perniit .coroparlsonn amons; Individual pupils, then 
appropriate d^a were produced. Hut' when tiieso testa were used lor other pur- 
|K)ses fiuch as to Becure a cienr «plcturc of what rending Kkills, a- particular 
Cblld iK>ssessid. then the resukiog,Uata zaay ^havc typically been more mlslcjtd- 
iBjjthan helpful. • 

Yet, because these tests have been .vrtdely Assed for so many yeariv and 
because they are produced by reputable comnjercial publishers (who distribute 
thorn with Jiost -of sophisticated mensurcuieiit trapplncs such, os technical^ 
* ftUablllty and validity reports), many.educators and most citizens nssume that^ 
. •tAndardlzed.aehievement test's are the ouir resi>cctablo instruments^one shouM 
Qle when atteniptmj^iu 4.nd out how well our schools are working, or more 
apectOcall^, just bow well an^lndividual pupil Is learnlnK*. 

. For purposes such as these, the^ use of a norm-reiorenced':tcst will olt en' pro- 
duce spurious data. And the tragedy is'that such data may be influential iti 
anrlTiae at far-jeachinj; decisions feparfiinp our nation's educational enter-- 
prisa l**or earpmple, scvernl recent report5'hav6 /o^ysjd ,cJn extensive "anfilyses ' 
©f the relative contribuUon of numerous factors lo'The quality of education. 
■ tThe results appeai; to be^disappoinlinx:. Tl^eaclier don't seem Xo make much of a ' 
. difference. Jnue^d.> schools th«mselves doil't seem to make, much of a dilTerence. 
Hot toiich of a difi^erence ^cUh respect to lohotf Invariably the Index pf pupil 
achievement used In ihesc^largcscale analyses has been performance on jiorm- . 
referenced testa.'And, as we shall see, there are characteristics ^f these meas* 
' ixret which rentier tJSeiti sufflciebtly Inappropriate 'for such analyses that the 
ttsnltlnj: data and subsequeht conclusions should be viewed witli great suspi- 
cion if not complete disd.ain. - ' ' , . ' , 

* ' * * DmcttKciES ny noBU-XEraitTrcn) tests 

trher* art two main -proble:* with typJccrl ^tandardired.tosts. which rcpder 
^ them unsuitable for widosCalc in assressinp the status of our children's edu- 
tatlofial attainments. These deiicits are assop'atcd ^vith the InterpreCabilitj*' 
and the psychomblric pVopertics^ of-nprm-teferenceB testa," ' - 

/fiforprciobUtVj/.—Most standardized testa arc dereloped by comjnereial Jest 
.pobllsbers who must desipn the instruments bo tliat they can effectively service 
an'tntlre nation. Pra-ctlcal economics preclude te3t publishers from developing . 
a 'separate test for New York and another version for Worthy Diikota, even 
though the^hjs'truttlonal emphases ofnhese two states may vary considerably. ' 
The'way that test publisl>ers pet out of this bind W\o develop n vcfy getural 
test which; while it may not be perfectly congruent with a ^:i^xn -school dis-^' 
•ttlct'a eurricvUat preference**, will at least, cover Bomc of them. .-J?ut to the 
.extent that a particular district is empliasizinK^ontont and skills other than 
those included in'the very broad standardized test, a misleading impression of- 
.the distrlcfi effectiveness or- an. individual child's capabilities may be created*' 
the use ot such' tests. 

Indeed, it it to tlic, advantspe of the commercial test publishers to keep 
achievement tests at very^generol- leyels, ^or' then educators throughout the. 
Mtion can derive the characteristic Rorschach , dividend ; they can usually see 
trhat they want to 'In an Ink blot. Thus, when eertnln teBt.s yield subscale 
•cores such a.s "reading eomprehdnsion;*' ^tJs inordinately difficult to get a pre- 
cise fix on what is^eant by that soorc. Only by dis.sectinj: the *cst itself can 
tbc user secure a-deftTislbtc Idea of whnt the Instrument is measuring. For. 
purposes such ns accurately locating. our nation's educationally dlsadvantaped 
youngsters, we need/more crisp interprptations than ai'b afforded by the bullc- 
©tnorm-refercnced tests, ♦ . - ^ 



ERLC 



Ktf<rrDCfd 
Ktgttv^od 



A more dftallfd trfntment of Iho. tJljtlntftnn^ hetWfen norm-reffrr nrtrt «tjd 
r IT^i - mfft»urfmft)t «roproAclir^. t^^ IVphnm. .W. J. (Hd ) '-tVlCfrlon- 
Jd'aitftVN^j^^gn'^" Inirodocuoo,*' irducaUooAl ixhnolocy ruUlctUoot. 
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statement by Popham (3) 



lost loafiine that by employinn a standnnUrcfl'ochlcveincnt tost -we had 
,<cd « cblld who Rcorcd Movr tiic tenth |H?rrcntIlc on a mntJjcmatlcs 
r ^ievcmcot test. Wc iinow. ot counjc. that wc^liuvc ft child %yho needs help io 
t&&tb. Jhit what kind ot help? The typienP Rcoreit on a standard irx^ znath 
achievement' test are oftca C»vcn In phrnses as cone nil as **haslc oiKjrntloha'^ or 
••ircoiDclrlc tclntlonshlpn.** With such imprccl«?c dcserlptora it h next to Impos* 
slt^le to realJj ideutif/ what the learner's wcukuesscs arc« much let>^ to correct 
thcra, ' ' 

Ptlfchomctrie propcri{€t, — As wc have seen, the chief purpose of norm-refer* 
e©cc<5 tests Is to i^rmlt compnriFons ^nionc Indlvid'uais, Because of this, such 
test* most produce xannnt pcorc?. Jn fact, the more that pupil Kcorea can be 
rprtsd out, tie tetter. Test items which ^re answered correctij bj most stn* 
dents, since they contribute little to total score variance, must be deleted or 
modincd. To contribute^ to total score variance nxr ideal item Is one which Is 
mnswered Correctly by half the people taking the. test (preferably those who 
«jK)re<l hiKhe^on the total test) and incorrectly by the other half (preferably 
Ihose vrbo scored lowest on the total testh Most standardized .tests \vhieh have 
been revls^ several times contain n great many sucJi Itclns Since, for purposes 
of J»preidJnjr out'tbostf Ukinc the test, these Itoius function eHectivcIy: But, in 
general, ipcb test items are most hicbly ^orreialcd with native -intellect tial 
olifitu- la other words, as standardized achievement tests are revised atfd* 
refined throucb the years in order to maximize the variability of pupil scores, 
tiltj mote and more ch>sely resemble a classic Intelllp:ence test. Tlius, norm-ref- 
erenced tests »re often quite insensitive td detecting the eflects of evea high 
quality fast ruction. - . ; 

^To Jllostrale, suppose a teacher attempts to teach an Important concept and, 
prior to instruction, administers a teSt item which almost jjveryone -misses. 
Yet, after a really fine iiistructibnal job, the same tesMtem is ans^vered cor* 
rectly'by cvCryQnc, But, because it produces no score variaace amoujr stud'ents, 
tbli kind of item would'have to be excluded from a standardized achievement 
test This not only leads to inscDsItlvc tests but creates' the further prpblem 
that oft-revised stsndafdlzed tests many times dp not contain th^' very test 
Items yrhich deal with' tHe central concepts of a Held. 

OoOTTducnoNa,BT CiaTEiiow-R£TEKE:?cE3) Tcaxs 

largely in tfa effort to remedy some of the wesknesses of norm-referenced 
laetgures, crlterlon*rcferenced tests arc dcsipied in such a way as to (1) be 
tnorc accurately iuterpretable. (2) yetect the effects of pood- instruction, and 
(8) allow us to make more accurate disposes of Individual learners' capabill- 
tlct<, , , . 

J)cflncd pupil compcfCTicic*.— ^nc of the important' ingredients of a well 
dcrised criterion •referenced test is an explicitly deilncd criterion. Putting jt 
another way, since the whole conception of this jiieasurcment stratepy is based 
on referencing scores to a criterion ect of learoci; behaviors, then the beba.vlors 



*tnutt be .described without amblpruUy. Most curreat criterionirefercnccd/mca'S' 
Orcment ppccialists arc advocating that a domain ot Icamei. behaviors be 
dellDcatCil in 'such a way th^t from the doraaln description (often called an 
Item form) an almost unlimited, numbci; of test items could 'be i^cnerated. It 
must be noted that ''test Item'* s^hbuld -bo conceived of a3 rqpreselitlnp a wide 
ntnge ^f nieasuremcut technlQtJcs, not mefcly paper and pencil tests* Because 
Of- the cbftmcterisiic accuracy of the criterion description^, we 1iSve a far 
better Idea of what it is that the student can or enn't 'do. This becomes parole-], 
Ularly imporlant when'. upon nssessnifr the students; wc discoverseridus (^t|ca- 
iional dcOclcnclfs. WJth a typical norni-referonoed test wc would h.ivc ohly a 
tlpbal Idea the' general sort of student wc.ikness; with a crilerion«rcfer- 
tnccd test the delicits qin be pini>olnted and thus more readily amelidrnted. " . 

£cnsitix*i(p to imiructton^ — 'Hecausc criterion-referenced tests fiec(L not pro- 
duce considerable scote varlnnce, they ciin consist even of items wrflch. aHer 
Instruction, most lenrners answer eorrrctly. Thoy cnii retain items which- are ■ 
l>aj^.^<^n the pVlnury curricula r cniphasisi As a conscnuence, such tests are, 
cfiorncjt.crlstically more sensitive than norm-referenced tests for punjoses.of 
dctccllnj: lustructloiml cOfectH. - ' 

Accvrotc diapnosit."-^\\ccii\\^c they arc more-carotully expltc^ite^d; criterioji- 
teferenced test^ typically provide us with ft more • nno-pnilncd amVysis 4>{ 
exactly what thc'^piipll can nnd can't^dcr. The tH^OTofentlal .skills we Tiope loam*.; 
enf^'Ul Acyuln? Cftn4»e more accurately .i>ortrftyi\l via a well dVservbiMl eritpri- " 
^cn*rrfcrfnccd test In, contrnst to Its oft(*d niporphous npHn-rcfcFeiieed couuter* 
part A ad foj promoti"ng lu.^tr actional improvcmuiit, accurate diagnosis is a a 
IndljjHjnssbn) ilrst . - ' • * h ♦ ' * 
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wiUTABotrrTEAcnnfOTOMCTcaT? - - 

DIiaiMfon« Koch as thc<?e often lead to the assertion that precisely expU* 
e»ted testa vriW cnccfumc<* Instructorn to tench to the test, nnd that Kuch n 
pr^ctlc^.ls Kornchow nprelicnslhlc. Contmrr to the wide-sprenrt MM that 
tetchlnfr to the test l» on Instrnclloual Mn, \vc must recojnilzc Ihnt if the tat 
it irvtu drfcnAibIc, thro we should nnplaud those who ca4i teach pupils to 
i tnantSrit. The kind of test xtblch will be <lefcnsi6lc Is not n particular set o( 
ltcxn^ however, but n iamplc Xrom hn ulniost Inpnlle number of Iti>m.^ that 
iCOuld be generated from our well described criterion. In other words, we 
.Should not be leaching to t J^Jven spt of 10- doublo-diCTt mulliplicnilon proh- 
IctDS, but lustead tp.cnj/ 8et of. 10 doubleKJIrrit mullipllcallon problems^rnn- 
domly wlecled trom a well defmed Hem ])ool. Thus the Jenmer acquires mas- 
tery of a cicu of skills, not a Jlmilcd n'mnber of items reflected by,n particular 
test Thla approach is central to.projjcr use of criterion-rcferen^Kl testing. 

* trtmiKO MOXET jl2»D lIKAStntlKO 8KIIX8 

- irb« irenenil thrust ot the Ie^risla1k)n^cnr^ently under consideration Involves, 
the dIst,rlbutIon of federal educational funds on. the basis of toensured educa~ 
tional deficiencies rather than census determiners.-t'^arther, there appears to be" 
% recopiltion of the ImporLnnce of. emploj-tep appropriate measurement metbo- 

^ doiOE^ when. IdentifyinR educationaily disadfantaped youngsters. Assuming 
Chat tufflclcDt care carf be taken to •suPi)ort the development of hich quatity 
criterton*rcferenced measures for.tbis purpose'r the general scbenie for target- 

^ lagfcdcn). dollars appears .to be sound.' For*when we are attempting .to }den- 
tif^. those youm; i)eople who truly need educational '^fsslstarice. 4hea ysing out- 
aitca census -enures as the determiner may b«e worse' than measuring iplleage* 
With a tablespoon. Ifs more-like measpring .baking poda with & speedometer^ 



'38B 



R€FERENCES 

Amick, D. J. and Wal berg, H.-J. (eds.): Introductory Multivariate 
Analysis for Educational Psychological, and Social Research. Berkeley. Ca . 
McCutchan Publishing Corporation, 1975. ' ~ 

•Berman, P. and McLaughlin, M. W.: Federal Programs Support'lnq Educational 
•Change . Vol. IVf The-h'nJjngs in IReview ; Santa Monica. Ca.: RAND. 1975. 

Blalock, H. M.: Social Statistics . New York: McGraw-Hill Co.", 1960. 

prophy, J. E. and.Evertson , C. M. : The Texas Teacher Effectiveness Project : 
Presentation of Non-linear Relationships and Summary Discussion . Report 
(lo.,.74-6.- Austin: Research and Development Center, University of T?xas, 
1974. 

, * " - * ' ' ■ ' 

Browrr; B'. .B. : The Expertinen'tal Mind In Education ; ■ New York: Harper and 

. Row, 1968. " ■ . ., 

- ■ . . - - ■ , « 

.t^oats,- W. D. ; Investigations and simulation of the relationships among 
selected classroom variables. Unpublished doctoral dissertation, University 
of Michigan, 1966. Cited by Flanders^ M.A. Analyzing TeBcher Behavior . 
Reading, Massachusetts: Addison-Wesley, J970. ^ ' . " 

Cochran, -W. G.. and Rubin, D. B.: Controllinq bias in observational studies;* 
A Review, SANKHYA , The Indian Journal of. Statistics , 1974, 35-A-, 1-30. 

C;arroll, J. B.: A model of school leamlnq. Teachers College Record , 1963, . 
64, 723-733. ^ ^ - 

Computing Services Office of University of 'Illfnblss SOUPAC Program 
Des^criptions t 1974., ' ' _ - . - ^ 

Deutsch, M". and. Associates. The Disadvantaged Chfld . ' New York: Basic. -Books., 
Inc. , 1967. . • •■ . , • • 

Gibbons, -M.: What isr individualized instruction? Iht ercHanqe ,- 197Q. 1 (2)", 
28-51, . ■ . ■ • , 

Glaser, R.: Instructional Technolffgy and- .the Measurement of learning outcomes 
Some Questions. American Psychologist , 1963.- 118 08). 519^521. " 

,Hays, W. L.: Statistics For Psychologists . 2nd. ed. New York: Holt, 1963." " 

HoUmah, ^S. and Boes, S.: Compensat6ry edu,cation: Hhat Works to Help 
disadvantaged Pupils . Arlington: National. School Public Relations Association 

W3: — — . _ ' • . , / 

Horn, J. L.: An empirical comparison of various methbds for computing factor 
scores. Educational and Psychological Measurement ^ 1965, 25, 313-322. 



VI-2 

389 



Hunt, 0. McV. Intelligence and experience . - New York: Ronald Press,- 1961. 

Jamison, D., Suppes, P. and Wells, s\ The effectiveness of alternate 
instructional media: a survey. Review of Educa tional Resea»*ch, 1974. ' 

44 (1), 1 --67. ;-: — 

Koehler, V. The California beginning teacher evaluation study. Performance 
Based Teacher Education , 1974, 2 (9) , 8 - 9. " , 

Lord, F. M. Some relations between Guttman's principal coihpo'nents of scale 
analysis and other psychometric theory. PSychometrika , 1958, 23, 291 - 296. 

Medley , D. M. , Soar, R. S. and Soar, R. M. Assessment and - research in teacher 
education : focus on PBTE . PBTE series no. 17, Washington, D. C.r Sierican . 
Association of ColTiges for Teacher Education, 1975. -, 

Mood,-A. M. Partitioning variance in multiple r.egression analyses. as a tool " 
for developing learning models. American Educational Resea rch Journal, 1371, 
.8 (2), 191 - 202. - - ~- ' — 

Soar, r; S. Optimum teacher-pupil interaction for pupil growth. ^Educational 
Leadership Research Supplement , 1968, 1 , 275 - 280. ^ 

Soar, R.- S. The Medley-Sojar-Toledo model, for research in teacher education, 
Gainesville, Florida: University of Florida, 1974 (mimeo). 

Soar, R. S. and Soar, R. M. An empirical analysis of selected Follow Through 
programs: an example of a process approach to evaluationT In I. J. Gordon (ed..) 
Early childhood education . Chicago: National Society far the Study of - - 
E3ucation; 1972.' \ ... 

Soar, R. S. and Soar^^ R. M. Classroom behavior , pupil characteristics and . ^ 
pupil growth for the school year and the summer . GainesviHe, 'Florida: 
Institute forlSeviTopment of HumarTResources, University of Florida, 1973. 

Tatsuoka^^M. M, Multivariate analysis in educational research." Chapter 9 in ^ . 
..F. N. Kerlinger (ed.). Review of research in . education -I. Itasca, Illinois:- ' 
-F. E. Peacock Publishers, Inc.,' 1973, '273 - ZW. ~^ 

Tatsuoka, M. M. Mu 1 1 i v a ri ate ana 1 ys i s : techniques for educaftional and - ' 
psychological research .:^ew York: Wiley, 1971. • ' " '° • 

Wargo,M. , ef a1 . Further examination of examplary programs for educating 
disadvantaged children. . Palo Alto, Catifornia :• American "Institute for 'c 
. Research, 197T. . - . ' ' • . /. - • ' 

White; S. H., et ?t1.. Federal programs' foryoung children : review and ' 
recommendations . Cambridge., Massachusetts: Huron .Institute, 1973. 

Wiley, D. E-. and Harnischfeger , A. Explosion of a mytli: quantity of . • 
schooling and exposure to instruction, major education vehicles. Educational 
Research, 1974,' 3 (4), , • • • 

Wilkerson, D; ' Compensatory education.: defining the- issues. " Chapter 2 in 
J. HellJtluth (ed.). Disadvantaged child . Vol. 3. New York: -Brunner/Mazel , ' 
Inc.', 1.970j.'24, -:35.. . ' 

yi-3 • . . -. 

3f)0 • ■ ^ • 



